RTView DataServer for Kafka Quick Start Guide, Version 5.3 (draft)

Owned by SL Docs
Last updated: Sept 25, 2020 by Diana Andrasi (Unlicensed)
8 min read

This document describes:

  • how to download and install the RTView DataServer for Kafka to be used with Enterprise Monitor
  • how to configure the data server including defining the connection information for the Solution Package for Kafka
  • how to add alert and historian database connections
  • how to add connections to Enterprise Monitor
  • how to verify that your setup is working properly in Enterprise Monitor.
For information regarding the overall RTView DataServer architecture as well as how to set up security for the related RTView processes, see DataServer and DataCollector Architecture for RTView Enterprise Monitor.
This flow assumes that you have already downloaded, installed, and have RTView Central Servers (Enterprise Monitor) up and running. See the RTView Enterprise Monitor Configuration Guide for more information.
If you are interested in setting up High Availability (failover) for your DataServer, review Configuring High Availability for DataServer Deliverables prior to starting this flow.
In addition to using the RTView Configuration Application for configuring your connections, you can use the Property Editor REST API to import initial connections or automate connection updates. See Property Editor REST API for more information.


Before Getting Started

  • Install Java installed and know the installation directory path.
  • Know the correct full path to the directory containing the Apache Kafka .jar files.
  • Know the settings for your Zookeeper connections: Client Port, Host, JMX Port, Username/Password, and Cluster Name.
  • Know the settings for your Broker connections: Client Port, Host, JMX Port, Username/Password, Cluster Name, and Topic Names.
  • Know the settings for your Producer connections: Host, JMX Port, Username/Password, and Cluster Name.
  • Know the settings for your Consumer connections: Host, JMX Port, Username/Password, Cluster Name, and Client-ID*.
For RTView to correctly associate the consumer JMX metrics with the other consumer metrics (for example, consumer lag), the connection name and the client ID need to be the same. You can either assign a connection name and use that name when you start your consumer process, or you can use the client ID (that is already running) as the connection name when adding connections.

If you are upgrading to a new version of the DataServer, review the Upgrade Steps to perform the required steps.

Downloading the RTView DataServer for Kafka

    1. Open a browser and navigate to https://sl.flexnetoperations.com.


    • 2. Log in using your RTView Download Center account login/password.
  • If you do not have an RTView Download Center account, contact sl@flexnetoperations.com to create an account.

    The RTView Download Center page displays:

3. Click the SL Corporation link in the Browse My Software and Documentation tab.

  • The list of available Product Lines displays:

4. Click RTView DataServer for Kafka.

  • The Product Information page displays:

5. Click RTView DataServer for Kafka.

  • The Product Download page displays:

6. Click the RTViewDataServerKafka_<version>.zip file or, if you want to download multiple files, select the check box next to the different deliverables and click the Download Selected Files button.

  • The compressed file(s) is/are saved to your machine.

7. Log in using your RTView Download Center account login/password.

Registration

When you install the DataServer, a temporary license key is included. SL Corporation will send you a permanent key that you will need to add to your RTView license key file, so that you can extend the operation of your monitoring system.

This is a universal key that will enable you to run RTView on different machines and platforms. Make sure you use RTView within the terms of your End User License Agreement (EULA).

To update your RTView system with your new license key:

  1. Open a command window and navigate to RTViewDataServerKafka/rtvapm/rtview/lib.

2. Edit the KEYS file using a text editor.

3. Copy/paste the new replacement key to the file.

4. Save the KEYS file.

Installing and Starting the RTView DataServer for Kafka

If using UNIX, do not include spaces in the installation directory path. The start_server.sh and stop_server.sh scripts will not function properly if spaces are included in the installation directory path.
  1. Extract All (Windows)/unzip (UNIX/Linux) the file to the directory of your choice. For UNIX/Linux, use unzip -a to unzip the file.

2. Open a command window and set the JAVA_HOME environment variable to point to your Java Installation.

EXAMPLE

export JAVA_HOME=/opt/Java/jdk1.7.0 (UNIX/Linux)

set JAVA_HOME=C:\Program Files (x86)\Java\jre1.8.0_201 (Windows)

Or, in Windows, you can set JAVA_HOME as an environment variable (in Control Panel > System >Advanced system settings), where it will then be used by all command windows and when you double-click on a .bat file.

3. In the command window, navigate to the RTViewDataServerKafka directory and type:

start_server.bat (Windows)

./start_server.sh (UNIX/Linux)

Or:

In Windows, you can double-click on start_server (Windows Batch File) in the RTViewDataServerKafka directory.

This script starts the Data Server on port 3278 with HTML Server (Eclipse Jetty, by default) on port 3270. If these ports conflict with other processes running on your system, you can change the first two digits of the ports by using -portprefix: and specifying the first two digits you want to use for the ports in the command line. For example: start_server.bat -portprefix:44 (which would change the ports to 4478 for the Data Server and 4470 for HTML Server). If you want to save your port prefix, you can change the port prefix in the RTView Configuration Application, or you can use -saveportprefix: command line argument (for example: start_server.bat -portprefix:44 -saveportprefix).

Configuring the DataServer

To specify the directory path to your Apache Kafka .jar files you need to create your connections using the RTView Configuration Application.

  1. Open a browser and type:

http://localhost:3270/rtvadmin

The Authentication required dialog displays:

2. Log in using rtvadmin/rtvadmin as the username/password.

The RTView Configuration Application displays:

3. Click RTView DataServer for Kafka.

The RTView Configuration Application displays with the General tab open:

If you modified the port at startup, you should modify the port prefix to avoid specifying the port on the command line every time you run the startup script.


To modify the port prefix:

  1. Navigate to Server Configuration > General > General Tab > Ports > Port Prefix and define a new port prefix. If you change the Port Prefix to 44, after saving your changes and restarting your data server, you would access the RTView Configuration Application for the RTView DataServer for Infrastructure using the following URL: http(s)://localhost:4470/rtvadmin (instead of the default http(s)://localhost:3270/rtvadmin URL)

2. Under Solution Package Configuration, select the solution package you want to configure and click the link below to view the steps required to set up the configuration for the selected solution package:

Apache Kafka

Data Cache: No configuration changes required

RTView Manager: No configuration changes required


Connecting Your Data Server to Enterprise Monitor

To connect your data server to Enterprise Monitor:

  1. Click Server Configuration > EM Integration.

The EM INTEGRATION tab displays:

2. Click the Integrate with Enterprise Monitor toggle.

By setting the RTVDS_INTEGRATEENABLED environment variable to 1 (for example: RTVDS_INTEGRATEENABLED=1) this field will be enabled by default.

The following fields are displayed:

Central Server Connection: Specify the URL to be used to connect to your RTView Enterprise Monitor Central Server.

By default, a local connection is defined (localhost:10018) and does not need to be modified if your Central Server and Data Server are on the same machine. If they are on different machines, enter the URL containing IP address and port:

<IP address>:10018

You can specify a different default for the Central Server Connection field by specifying the RTVDS_CENTRALURL environment variable (for example: RTVDS_CENTRALURL=RTViewCentralHost:10018.

RTView Data Server Connection: Specify the URL that the RTView Enterprise Monitor Central Server should use for connecting to this RTView Data Server. The default value is the internal IP address. If the Central Server cannot access the internal IP address, then you must modify this value to use the external IP address. 

This field can be set up by defining the RTVDS_URL environment variable (for example: RTVDS_URL=myhost:3278).

RTView Data Server Query URL: Specify the URL that the RTView Central Server HTML user interface should use to query data from this RTView Data Server. The default value uses the internal IP address. If all browser clients cannot access the internal IP address for this system, you must change this value to use the external IP address or deploy the rtvquery servlet to an application server on an accessible system. You should specify that value in this field. 

This field can be set up by defining the RTVDS_QUERYURL environment variable (for example: RTVDS_QUERYURL=http://myhost:3270/rtvquery).

RTView Data Server Name: Define the name of the data server that you want to display in Enterprise Monitor. Modifying the default name in this field is optional. However, for example, enter Kafka_Data_Server to verify the change in Verifying Your Setup section. 

This field can be set up by defining the RTVDS_NAME environment variable (for example: RTVDS_NAME=KAFKAMON_LOCAL).

You can only use alphanumeric characters, underbars, and dashes when naming the data server.

CI Types to Exclude: Use this field if you want to exclude CI Types from the Service Model in the Central Server. If not specified, all CI Types for the configured solution packages are included in the Service Model. To select the CI Types you want to exclude, click the  button to display the CI Types to Exclude dialog. Then select the desired CI Types, and click OK. 

This field can be set up by defining the RTVDS_CITYPESTOEXCLUDE environment variable (for example: RTVDS_CITYPESTOEXCLUDE=KAFKA-BROKER,KAFKA-CONSUMER).

Send Connection Information To Enterprise Monitor: By default, the associated Announce this RTView Data Server to  the Central Server toggle is selected, which will automatically connect the data server to RTView Central. If you do not want to automatically connect the data server to RTView Central, deselect this toggle. If deselected, you must define the connection from RTView Central to this data server in the RTView Central Configuration Application.

This field can be defined by setting the RTVDS_ANNUNCIATEENABLED environment variable to 1 (for example: RTVDS_ANNUNCIATEENABLED=1).

Saving Your Changes and Restarting The DataServer

  1. Once you have created all of your connections, click the SAVE button in the RTView Configuration Application.

2. For your changes to go into effect, you must stop and restart the dataserver from your installation directory by typing the following in your command window:

stop_server.bat (Windows)

./stop_server.sh (UNIX/Linux)

start_server.bat (Windows)

./start_server.sh (UNIX/Linux)

Or:

Click the  button in the upper right-hand corner.

Verifying Your Setup

To verify your setup:

  1. Open a browser, access RTView Configuration Application for RTView Central Servers (Enterprise Monitor) using http://localhost:10070/rtview-central-rtvadmin, and log in using rtvadmin/rtvadmin as the username/password.

Your data server (Kafka_Data_Server, for example) displays in the list below RTView Central Servers. You can edit your data server connections/configurations in the RTView Configuration Application for  RTView Central (http://localhost:10070/rtview-central-rtvadmin) or in the RTView Configuration Application for the RTView DataServer for Kafka deliverable (http://localhost:3270/rtvadmin).

2. Open a browser, access Enterprise Monitor (for example, http://localhost:10070/rtview-central-classic), and log in using admin/admin as the username/password.

3. Click Admin > Architecture > System Overview and make sure that your newly connected data server (Kafka_Data_Server, for example) displays in the Data Servers tab and that CI Metrics are being collected.

4. Once you have set up your connections for your data server, you can verify the setup by accessing the following URL:

http(s)://localhost:3270/common

The RTView Cache Viewer application displays showing details for the caches that are collecting data: