Home

OpsMx Tcollector Wiki

*Below are the observed points that we can tweek the Tcollector

1. hbase_master.py

   • The hbase_master.py script can generates around 1500 metrics. If we need only some context metrics types, we can specify in habase_master.py with EXCLUDED_CONTEXTS variable. The context(Metrics type) which are specified in EXCLUDED_CONTEXTS are "excluded". The possible context metrics types given below

   masterstatistics
   concurrentmarksweep
   rpcstatistics-33985
   mapped
   regionserverdynamicstatistics
   par_eden_space
   regionserverstatistics
   cms_perm_gen
   par_survivor_space
   codecachemanager
   cms_old_gen
   code_cache
   direct
   rpcstatistics-33449
   

2. mysql_opsmx.py

   • Set extend=True in opsmxconf.py(MYSQL_CONFIG Dictionary) to get more number of metrics.Check opmx_mysql metrics page for info

3. tcollector.py

   • If the collector's datapoints are repeatedly getting same value, the tcollector.py sends the same metrics for every 5 Min regardless of COLLECTER_TIME_INTERVAL
   • By default OpenTSDB http API support 64 data points at a time. If we want to enable/disable chunking the metrics, there is a field "chunking" in 'tcollector/collectors/etc/config.py'
   • dedupinterval - Number of seconds in which successive duplicate datapoints are suppressed before sending to the TSD. Use zero to disable. default=300. Set 0 for dedupinterval in tcollector/collectors/etc/config.py to disable deduplication
   • Run the collector script directly for testing

     sudo PYTHONPATH=`pwd` python collectors/0/procstats.py
     

4. URL Redirect & HTTPS Proxy

   • URL Redirect

     Must enable proxy module in apache. Refer Apache ProxyPassMatch Documentation for more info on ProxyPassMatch

     Config File: /etc/apache/sites-available/000-default.conf(File name may change in your machine)

     NOTE:

     • The TSDB API ProxyPassMatch should be bottom of all other ProxyPassMatch like below example
     • If you enable HTTPS, please specify ProxyPassMatch configs in HTTPS section also.

     # Below are OpsMx custom APIs
     ProxyPassMatch "/opsmx-analysis/*" "http://localhost:8161" 
     ProxyPassMatch "/opsmx-cas-services/*" "http://localhost:8161"
     
     ProxyPassMatch "/opsmx-analysis/*" "http://localhost:8161"
     ProxyPassMatch "/cas/*" "http://localhost:8090"
     ProxyPassMatch "/canaries/*" "http://localhost:8090"
     ProxyPassMatch "/auth/*" "http://localhost:8090"
     ProxyPassMatch "/registerCanary/*" "http://localhost:8090"
     
     # Below URL is Open TSDB HTTP API
     ProxyPassMatch "/api/*" "http://127.0.0.1:4343"
     

   • HTTPS Config

     • Please refer this blog post for HTTPS config (Use only in developer's environment, but not on production!)
     • For free HTTPS certificate generation, please refer Let's Encrypt and certbot

5. *DEPENDENCIES for tcollector

   • sudo apt-get install python-pip python-dev libmysqlclient-dev
   • pip install MySQL-python (For mysql_opsmx.py agent)
   • pip install psycopg2 (For postgress_opsmx.py agent)
   • pip install pymongo (For mongo.py agent)