- Published on
Using Apache Thrift 1: Motivation
- Authors
- Name
- Daisuke Kobayashi
- https://twitter.com
Over the next several posts, I am going to write about Apache Thrift. Apache Thrift is an RPC (Remote Procedure Call) framework developed by Facebook for scalable cross-language service development. Services that use Thrift include Facebook, the Evernote API, and Apache Cassandra. With this framework, you can implement RPC across different programming languages with relative ease.
I became interested in distributed processing about three months ago and started using Thrift. At the time I set it up on Windows with Visual C++, but the documentation was sparse and it took a lot of effort. More recently, a friend of mine also tried to set it up and ran into the same issues, so I decided to write about it.
Motivation
The equipment I work on is a fairly large system and requires high-speed image processing. To make it faster, multiple PCs inside the machine are connected over a LAN, and heavy computation is offloaded to dedicated compute PCs so that the load can be distributed. That naturally led me to take an interest in distributed processing.
Our product is a legacy system and extremely complex. The implementation uses socket-based TCP/IP communication to transfer commands and data and have remote machines process them. I had almost no experience with network programming at the time, so building a distributed system from scratch with sockets felt like a very long road. I started looking around for something that would make development easier.
That led me to Google Protocol Buffers, Apache Thrift, REST, and other options. I really like RESTful ideas because they feel close to web standards. However, the system I had in mind was closed, so Thrift seemed like the better fit and I started using it. I have mainly used it for small tools, and it has been very convenient. In the next post, I will cover the environment setup.
Reference
This page gives a good overview of several distributed-processing technologies. http://www.atmarkit.co.jp/fjava/special/distributed02/distributed02_1.html