还有很多与kafka集成的外部的工具。更多信息点击这里，包含了stream处理系统，hadoop的集成，监控和部署工具。

Here is a list of tools we have been told about that integrate with Kafka outside the main distribution. We haven't tried them all, so they may not work!

Clients, of course, are listed separately here.

Starting with the 0.8 release we are maintaining all but the jvm client external to the main code base. The reason for this is that it allows a small group of implementers who know the language of that client to quickly iterate on their code base on their own release cycle. Having these maintained centrally was becoming a bottleneck as the main committers can't hope to know every possible programming language to be able to perform meaningful code review and testing. This lead to a scenario where the committers were attempting to review and test code they didn't understand.

We are instead moving to the redis/memcached model which seems to work better at supporting a rich ecosystem of high quality clients.

We haven't tried all these clients and can't vouch for them. The normal rules of open source apply.

If you are aware of other clients not listed here (or are the author of such a client), please add it here. Please also feel free to help fill out information on the features the client supports, level of activity of the project, level of documentation, etc.

-------------

Robust high performance C/C++ library with full protocol support

A bunch of other language bindings has been built on top of it, including Haskell, Node.js, OCaml, PHP, Python, Ruby, C# / .NET.

 

Kafka Version: 0.7.x, 0.8.x, 0.9.x, 0.10.x

Maintainers: Magnus Edenhill

-------------

Native C++ library with protocol support for Metadata, Produce, Fetch, and Offset.

Kafka Version: 0.8.x

Maintainer: David Tompkins

License: Apache 2.0

 

-------------

C++ Header-only Kafka Client Library using Boost Asio

 

Kafka Version: 0.8.x

Maintainer: Daniel Joos

License: MIT

 

-------------

A C++11 asyncronous producer/consumer library for Apache Kafka based on boost asio

Kafka Version: 0.8.x

Maintainer: Svante Karlsson

License: Boost Software License - Version 1.0

 

-------------

Kafka Version: 0.8.x

 

-------------

Kafka Version: 0.7.x

 

-------------

-------------

High performance Python client based on Librdkafka with full protocol support.

Docs: http://docs.confluent.io/current/clients/index.html

Kafka Version: 0.8.x, 0.9.x, 0.10.x

Maintainers: Confluent

License: Apache 2.0

 

-------------

Pure Python implementation with full protocol support. Consumer and Producer implementations included, GZIP, LZ4, and Snappy compression supported.

Kafka Version: 0.8.x, 0.9.x, 0.10.x

Maintainer: Dana Powers

License: Apache 2.0

 

-------------

Python driver with full protocol support and balanced consumer implementation. GZIP and Snappy compression supported.

Kafka Version: 0.8.x

Maintainer: Parse.ly

License: Apache v2.0

 

-------------

Protocol support for Kafka 0.7 in Python. GZip and Snappy compression supported

Kafka Version: 0.7.x

Maintainer: David Arthur

License: Apache v.2.0

-------------

Also:

 

Kafka Version: 0.7.x

Maintainer:: Dan Sulley, LinkedIn

License: Apache 2.0

-------------

Python client from Disqus:

 

Kafka Version: 0.7.x

Maintainer:: Keith Bourgoin, Parse.ly

License: Apache 2.0

-------------

Python client from Urban Airship: https://github.com/urbanairship/pykafkap

 

Kafka Version: 0.7.x

-------------

Python client from Datadog: https://github.com/datadog/brod

(Producer, Simple Consumer, ZK-Consumer)

 

Kafka Version: 0.7.x

-------------

-------------

Pure Go implementation with full protocol support. Consumer and Producer implementations included, GZIP and Snappy compression supported.

Kafka Version: 0.8.x

License: MIT

 

-------------

Enhanced Go Kafka consumer and producer implementations.

Kafka Version: 0.8.x

License: Apache v2.0

 

-------------

A pure Go implementation of the low level Kafka API.

Kafka Version: 0.8.x

License: Apache v2.0

 

-------------

A fast pure Go Kafka implementation with a clean API.

 

Kafka Version: 0.8.x

License: MIT

-------------

Kafka Version: 0.7.x

-------------

confluent-kafka-go: Confluent's Kafka client for Golang wraps the librdkafka C library, providing full Kafka protocol support with great performance and reliability.

The Golang bindings provides a high-level Producer and Consumer with support for the balanced consumer groups of Apache Kafka 0.9 and above.

Kafka Version: 0.8.x+

License: Apache v2.0

Docs: http://docs.confluent.io/current/clients/index.html

-------------

-------------

Kafka client library in Erlang. Full support for 0.9+ consumer protocol, very efficient producer implementation.

 

Kafka Version: 0.9.x, 0.10.x

Maintainers: Ivan Dyachkov (Klarna AB), Shi Zaiming (Klarna AB)

License: Apache 2.0

 

-------------

A minimal, high-performance Kafka client in Erlang

Kafka Version: 0.8.x

Maintainer: Helpshift

License: Apache v2

 

-------------

erlkafka is a kafka client written in erlang

Kafka Version: 0.7.x

Maintainer: Milind Parikh

*License: BSD, LGPL

Also:

 

-------------

A fully featured .NET client for Apache Kafka based on librdkafka (a fork of rdkafka-dotnet).

Kafka Version: 0.8.x - 0.10.x

Maintainer: Confluent Inc. (original author Andreas Heider)

License: Apache 2.0

 

-------------

Pure C# client with full protocol support.  Includes consumer, producer,

lower level components and gzip support (no snappy)

Kafka Version: 0.8.x

Maintainer: James Roland 

License: Apache 2.0

 

-------------

This is a .NET implementation of a client for Kafka using C# for Kafka 0.8. It provides for an implementation that covers most basic functionalities to include a simple Producer and Consumer.

Kafka Version: 0.8.x

Maintainer: ExactTarget

License: Apache 2.0

.Net implementation of the Apache Kafka Protocol that provides basic functionality through Producer/Consumer classes. The project also offers balanced consumer implementation. The project is a fork from ExactTarget's Kafka-net Client.

 

Kafka Version: 0.8.x, 0.9.x

Maintainer: Microsoft

License: Apache 2.0

 

-------------

C# client, asynchronous, all 3 compressions supported (read and write), tracks leader partition changes transparently, long time in production.

 

Kafka Version: 0.8.x

Maintainer: Vadim Chekan

License: Apache-2.0

 

-------------

kafka-sharp - "High Performance" .NET Kafka Driver

 

Kafka Version: 0.8.x

Maintainer: Criteo

License: Apache 2.0

 

-------------

 

-------------

Kafka Version: 0.8.x

Kafka Version: 0.8.x

https://github.com/pingles/clj-kafka/ 

 

-------------

Kafka Version: 0.8.x

Code, Documentation

-------------

 

-------------

[Unmaintained] Pure Ruby, Consumer and Producer implementations included, GZIP and Snappy compression supported. Ruby 1.9.3 and up (CI runs MRI 2.0, JRuby and Rubinius).

Kafka Version: 0.8.x

Maintainer: Bob Potter

License: MIT

 

-------------

Karafka - Framework used to simplify Apache Kafka based Ruby applications development.

 

Kafka Version: 0.9.x

Maintainer: Maciej Mensfeld

Licence: MIT

 

-------------

ruby-kafka - A Ruby client library for the Kafka distributed log system

Kafka Version: 0.8.x

Maintainer: Zendesk/ Daniel Schierbeck

Licence: Apache 2.0

-------------

 

Kafka Version: 0.8.x

Maintainer: Joseph Lawson

License: Apache 2.0

 

-------------

 

Kafka Version: 0.7.x

Maintainer: Alejandro Crosa

License:: Apache 2.0

-------------

Event machine client:

 

Kafka Version: 0.7.x

-------------

JRuby Event stream processor

 

Kafka Version: 0.7.x

-------------

 

 

-------------

The node-rdkafka library is a high-performance NodeJS client for Apache Kafka that wraps the native librdkafka library. All the complexity of balancing writes across partitions and managing (possibly ever-changing) brokers should be encapsulated in the library.

Kafka Version: 0.9, 0.10

Maintainer: Blizzard.com

License: MIT

 

 

-------------

Kafka-Node is a NodeJS client with Zookeeper integration

 

Kafka Version: 0.8.x

Maintainer: sohu.com

License: MIT

 

-------------

Kafka-node is a pure JavaScript implementation for NodeJS Server with Vagrant and Docker support.

Kafka Version: 0.8.x

Maintainer: wurstmeister

License: Apache 2.0

 

-------------

Node-kafka is a node.js wrapper for the C library librdkafka

 

Kafka Version: 0.8.x

Maintainer: Sutoiku

License: MIT

 

-------------

kafka-java-bridge is a Nodejs wrapper for the JAVA high level kafka 0.8. consumer API

 

Kafka Version: 0.8.x

Maintainer: LivePersonInc

License: MIT

 

-------------

Low-level protocol support in node.js.

 

Kafka Version: 0.7.x

Maintainers: 

License: MIT

 

-------------

Alternate node client from Tagged

Also:

 

Kafka Version: 0.7.x

-------------

 

-------------

Dory is a producer daemon that supports clients in various programming languages.  Clients send messages to Dory using local interprocess communication.  Dory then takes full responsibility for reliable message delivery.  Monitoring infrastructure can query Dory's web interface for JSON-based status and data quality reports.

 

Kafka Version: 0.8.x

Maintainer: Dave Peterson

License: Apache v.2.0

-------------

Kafka Version: 0.8.x

Maintainer: Big Data Open Source Security LLC

License: Apache 2.0

 

-------------

(Deprecated) Kafka high level Producer and Consumer APIs are very hard to implement right. Rest endpoint gives access to native Scala high level consumer and producer APIs.

 

Maintainer: Sasha Klizhentas

License: Apache 2.0

 

-------------

The Kafka REST Proxy provides a RESTful interface to a Kafka cluster.

 

Kafka Version: 0.8.x, 0.9.x, 0.10.x

Maintainer: Confluent 

License: Apache 2.0

 

-------------

Kafka Version: 0.8.x

Maintainer: Mailgun

License: Apache 2.0

 

-------------

Efficient Kafka REST Proxy for producers

 

Kafka Version: 0.8.x

Maintainers: Ivan Dyachkov (Klarna AB), Shi Zaiming (Klarna AB)

License: Apache 2.0

 

 

-------------

Pure Perl, Consumer and Producer implementations included. Zookeeper

integration. GZIP and Snappy compression supported.

 

Kafka Version: 0.8.x

Maintainer: Sergey Gladkov

License: Artistic License

 

-------------

Generic producer and consumer for stdin and stdout.

Kafka Version: 0.8.x

Maintainer: Magnus Edenhill

License: 2-clause BSD

 

-------------

PHP extension built on librdkafka

 

Kafka Version: 0.8.x

Maintainer:  Elias Van Ootegem

License: MIT

 

 

-------------

PHP client based on librdkafka

Kafka Version: 0.8.x

Maintainer: Arnaud Le Blanc, for Mention.com

License: MIT

 

 

-------------

PHP library with Consumer (simple and Zookeeper-based), Producer and compression support (release notes).

 

Kafka Version: 0.7.x

Maintainer: Lorenzo Alberton

License: Apache v.2.0

Also:

Log4PHP Appender

 

-------------

 

-------------

Pure Rust implementation with support for Metadata, Produce, Fetch, and Offset requests. Supports Gzip and Snappy compression

 

Kafka Version: 0.8.x

Maintainer:  Yousuf Fauzan

License: MIT 

 

-------------

Of course the main project maintains a set of jvm-based clients. But here are alternate clients.

Krackle is an optimized Kafka client built by Blackberry.

 

Kafka Version: 0.8.x

Maintainer: Blackberry

License: Apache 2.0

 

-------------

Port of Apache storm-kafka to >= 0.8.0

Maintainer: wurstmeister

License: Apache 2.0

 

-------------

A DSL for Scala developers to produce and consume messages with Kafka.

 

Kafka Version: 0.8.x

Maintainer: Big Data Open Source Security LLC

License: Apache 2.0

 

-------------

 

Kafka Version: 0.7.x

 

2 people like this

When is the python client for 0.8.x complete? _