Configuring Apache Kafka for DataCollector, Version 5.1.1

This section describes the required configurations as well as the optional configurations available for the Solution Package for Apache Kafka in the RTView Configuration Application. You must define the classpath to the Apache Kafka jar files and you must also define data source connections for each connection that you want to monitor. 

Configuring Data Collection

  • Click Solution Package Configuration > Apache Kafka.

The CONNECTIONS tab displays.

Note: No setup is required for Solution Package Configuration > RTView Manager or Data Cache.

  • Enter the correct full path to the directory containing the Apache Kafka jar files in the Classpath field.
  • Click the  button.

The Add Connection dialog displays.

  • Add your desired connections and click the SAVE button after each new connection. You must add at least one Zookeeper and one Broker connection.

Broker Connection

Connection Name: The name of the connection/server.

Connection Type: Select Broker from this drop down list.

Host: The IP address of the host.

JMX Port: The JMX port used when connecting.

Username: The username is used when creating the connection. This field is optional (if no username/password is set for the broker).

Password: This password is used when creating the connection. This field is optional (if no username/password is set for the broker). By default, the password entered is hidden. Click the associated "eye" icon to view the password text.

Cluster Name: The name of the cluster.

Topic Names: Optionally enter one or more topics to restrict monitoring to only those topics on the broker (all topics are monitored by default). You can enter multiple topics by adding a comma or clicking the Tab key after each. Once entered, you can click the X next to the topic name to remove them.

Client Port: Optionally enter the Client Port if you want to use this broker to monitor topic partitions for this cluster.

Consumer Connection

Connection Name: The name of the connection/server.

Connection Type: Select Consumer from this drop down list.

Host: The IP address of the host.

JMX Port: The JMX port used when connecting.

Username: The username is used when creating the connection. This field is optional (if no username/password is set for the consumer).

Password: This password is used when creating the connection. This field is optional (if no username/password is set for the consumer). By default, the password entered is hidden. Click the associated "eye" icon to view the password text.

Cluster Name: The name of the cluster.

Producer Connection

Connection Name: The name of the connection/server.

Connection Type: Select Producer from this drop down list.

Host: The IP address of the host.

JMX Port: The JMX port used when connecting.

Username: The username is used when creating the connection. This field is optional (if no username/password is set for the producer).

Password: This password is used when creating the connection. This field is optional (if no username/password is set for the producer). By default, the password entered is hidden. Click the associated "eye" icon to view the password text.

Cluster Name: The name of the cluster.

Zookeeper Connection

Connection Name: The name of the connection/server.

Connection Type: Select Zookeeper from this drop down list.

Client Port: Optionally enter the Client Port if you want to monitor topic partitions for this cluster.

Host: The IP address of the host.

JMX Port: The JMX port used when connecting.

Username: The username is used when creating the connection. This field is optional (if no username/password is set for the zookeeper).

Password: This password is used when creating the connection. This field is optional (if no username/password is set for the zookeeper). By default, the password entered is hidden. Click the associated "eye" icon to view the password text.

Cluster Name: The name of the cluster.

  • You can optionally modify the Poll Rates (query interval, in seconds) that will be used to collect the metric data for various general and disk usage caches in Solution Package Configuration > Apache Kafka DATA COLLECTION > Poll Rates.