|
| 1 | +Kafka |
| 2 | +===== |
| 3 | + |
| 4 | +By enabling this toolset, HolmesGPT will be able to fetch metadata from Kafka. This provides Holmes |
| 5 | +the ability to introspect into Kafka by listing consumers and topics or finding lagging consumer groups. |
| 6 | + |
| 7 | +This toolset uses the AdminClient of the |
| 8 | +`confluent-kafka python library <https://docs.confluent.io/platform/current/clients/confluent-kafka-python/html/index.html#pythonclient-adminclient>`_. |
| 9 | +Kafka's `Java API <https://docs.confluent.io/platform/current/installation/configuration/admin-configs.html>`_ |
| 10 | +is also a good source of documentation. |
| 11 | + |
| 12 | +Configuration |
| 13 | +------------- |
| 14 | + |
| 15 | +.. code-block:: yaml |
| 16 | +
|
| 17 | + holmes: |
| 18 | + toolsets: |
| 19 | + kafka/admin: |
| 20 | + enabled: true |
| 21 | + config: |
| 22 | + kafka_clusters: |
| 23 | + - name: aks-prod-kafka |
| 24 | + kafka_broker: kafka-1.aks-prod-kafka-brokers.kafka.svc:9095 |
| 25 | + kafka_username: kafka-plaintext-user |
| 26 | + kafka_password: ****** |
| 27 | + kafka_sasl_mechanism: SCRAM-SHA-512 |
| 28 | + kafka_security_protocol: SASL_PLAINTEXT |
| 29 | + - name: gke-stg-kafka |
| 30 | + kafka_broker: gke-kafka.gke-stg-kafka-brokers.kafka.svc:9095 |
| 31 | + kafka_username: kafka-plaintext-user |
| 32 | + kafka_password: **** |
| 33 | + kafka_sasl_mechanism: SCRAM-SHA-512 |
| 34 | + kafka_security_protocol: SASL_PLAINTEXT |
| 35 | +
|
| 36 | +Below is a description of the configuration field for each cluster: |
| 37 | + |
| 38 | +.. list-table:: |
| 39 | + :header-rows: 1 |
| 40 | + :widths: 20 80 |
| 41 | + |
| 42 | + * - Config key |
| 43 | + - Description |
| 44 | + * - name |
| 45 | + - Give a meaningful name to your cluster. Holmes will use it to decide what cluster to look into. Names must be unique across all clusters. |
| 46 | + * - kafka_broker |
| 47 | + - List of host/port pairs to use for establishing the initial connection to the Kafka cluster. Comma separated values. |
| 48 | + * - kafka_client_id |
| 49 | + - An id string to pass to the server when making requests. The purpose of this is to be able to track the source of requests beyond just ip/port by allowing a logical application name to be included in server-side request logging. |
| 50 | + * - kafka_security_protocol |
| 51 | + - Protocol used to communicate with brokers. Valid values are: PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL |
| 52 | + * - kafka_sasl_mechanism |
| 53 | + - SASL mechanism used for client connections. This may be any mechanism for which a security provider is available. GSSAPI is the default mechanism. |
| 54 | + * - kafka_username |
| 55 | + - Sets the SASL credentials used for this client. This key sets the username. |
| 56 | + * - kafka_password |
| 57 | + - Sets the SASL credentials used for this client. This key sets the password. |
| 58 | + |
| 59 | +.. include:: ./_toolset_configuration.inc.rst |
| 60 | + |
| 61 | +Capabilities |
| 62 | +------------ |
| 63 | +.. include:: ./_toolset_capabilities.inc.rst |
| 64 | + |
| 65 | +.. list-table:: |
| 66 | + :header-rows: 1 |
| 67 | + :widths: 30 70 |
| 68 | + |
| 69 | + * - Tool Name |
| 70 | + - Description |
| 71 | + * - list_kafka_clusters |
| 72 | + - Lists all configured Kafka clusters |
| 73 | + * - list_kafka_consumers |
| 74 | + - Lists all Kafka consumer groups in the cluster |
| 75 | + * - describe_consumer_group |
| 76 | + - Describes a specific Kafka consumer group |
| 77 | + * - list_topics |
| 78 | + - Lists all Kafka topics in the cluster |
| 79 | + * - describe_topic |
| 80 | + - Describes details of a specific Kafka topic |
| 81 | + * - find_consumer_groups_by_topic |
| 82 | + - Finds all consumer groups consuming from a specific topic |
0 commit comments