SUMMARY.md

Table of contents

• Introduction

Basics

• Concepts
• Architecture
• Components
  • Cluster
  • Controller
  • Broker
  • Server
  • Minion
  • Tenant
  • Schema
  • Table
  • Segment
  • Deep Store
  • Pinot Data Explorer
• Getting Started
  • Running Pinot locally
  • Running Pinot in Docker
  • Quick Start Examples
  • Running in Kubernetes
  • Running on public clouds
    • Running on Azure
    • Running on GCP
    • Running on AWS
  • Batch import example
  • Stream ingestion example
  • HDFS as Deep Storage
  • Troubleshooting Pinot
  • Frequently Asked Questions (FAQs)
    • General
    • Pinot On Kubernetes FAQ
    • Ingestion FAQ
    • Query FAQ
    • Operations FAQ
• Import Data
  • From Query Console
  • Batch Ingestion
    • Spark
    • Hadoop
    • Backfill Data
    • Dimension Table
  • Stream ingestion
    • Apache Kafka
    • Amazon Kinesis
    • Apache Pulsar
  • Stream Ingestion with Upsert
  • Stream Ingestion with Dedup
  • File Systems
    • Amazon S3
    • Azure Data Lake Storage
    • HDFS
    • Google Cloud Storage
  • Input formats
  • Complex Type (Array, Map) Handling
• Indexing
  • Forward Index
  • Inverted Index
  • Star-Tree Index
  • Bloom Filter
  • Range Index
  • Native Text Index
  • Text search support
  • JSON Index
  • Geospatial
  • Timestamp Index
• Releases
  • 0.11.0
  • 0.10.0
  • 0.9.3
  • 0.9.2
  • 0.9.1
  • 0.9.0
  • 0.8.0
  • 0.7.1
  • 0.6.0
  • 0.5.0
  • 0.4.0
  • 0.3.0
  • 0.2.0
  • 0.1.0
• Recipes
  • GitHub Events Stream

For Users

• Query
  • Querying Pinot
  • Aggregation Functions
  • Transformation Functions
  • User-Defined Functions (UDFs)
  • Grouping Algorithm
  • Query Options
  • Cardinality Estimation
  • Lookup UDF Join
  • Querying JSON data
  • Filtering with IdSet
  • Explain Plan
  • GapFill Function For Time-Series Dataset
• APIs
  • Broker Query API
    • Query Response Format
  • Controller Admin API
• External Clients
  • JDBC
  • Java
  • Python
  • Golang
• Tutorials
  • Use OSS as Deep Storage for Pinot
  • Ingest Parquet Files from S3 Using Spark
  • Creating Pinot Segments
  • Use S3 as Deep Storage for Pinot
  • Use S3 and Pinot in Docker
  • Batch Data Ingestion In Practice
  • Schema Evolution

For Developers

• Basics
  • Extending Pinot
    • Writing Custom Aggregation Function
    • Segment Fetchers
  • Contribution Guidelines
  • Code Setup
  • Code Modules and Organization
  • Update Documentation
• Advanced
  • Data Ingestion Overview
  • Ingestion Aggregations
  • Ingestion Transformations
  • Null Value Support
  • V2 Multi-Stage Query Engine
  • Advanced Pinot Setup
• Plugins
  • Write Custom Plugins
    • Input Format Plugin
    • Filesystem Plugin
    • Batch Segment Fetcher Plugin
    • Stream Ingestion Plugin
• Design Documents
  • Segment Writer API

For Operators

• Deployment and Monitoring
  • Setup cluster
  • Setup table
  • Setup ingestion
  • Decoupling Controller from the Data Path
  • Segment Assignment
  • Instance Assignment
  • Rebalance
    • Rebalance Servers
    • Rebalance Brokers
  • Tiered Storage
  • Pinot managed Offline flows
  • Minion merge rollup task
  • Consistent Push and Rollback
  • Access Control
  • Monitoring
  • Tuning
    • Realtime
    • Routing
  • Upgrading Pinot with confidence
• Command-Line Interface (CLI)
• Configuration Recommendation Engine
• Tutorials
  • Authentication, Authorization, and ACLs
  • Configuring TLS/SSL
  • Build Docker Images
  • Running Pinot in Production
  • Kubernetes Deployment
  • Amazon EKS (Kafka)
  • Amazon MSK (Kafka)
  • Monitor Pinot using Prometheus and Grafana
  • Performance Optimization Configurations

Configuration Reference

• Cluster
• Controller
• Broker
• Server
• Table
• Schema
• Ingestion Job Spec
• Monitoring Metrics
• Functions
  • ABS
  • ADD
  • arrayConcatDouble
  • arrayConcatFloat
  • arrayConcatInt
  • arrayConcatLong
  • arrayConcatString
  • arrayContainsInt
  • arrayContainsString
  • arrayDistinctInt
  • arrayDistinctString
  • arrayIndexOfInt
  • arrayIndexOfString
  • ARRAYLENGTH
  • arrayRemoveInt
  • arrayRemoveString
  • arrayReverseInt
  • arrayReverseString
  • arraySliceInt
  • arraySliceString
  • arraySortInt
  • arraySortString
  • arrayUnionInt
  • arrayUnionString
  • AVGMV
  • Base64
  • ceil
  • CHR
  • codepoint
  • concat
  • count
  • COUNTMV
  • day
  • dayOfWeek
  • dayOfYear
  • DISTINCT
  • DISTINCTCOUNT
  • DISTINCTCOUNTBITMAP
  • DISTINCTCOUNTBITMAPMV
  • DISTINCTCOUNTHLL
  • DISTINCTCOUNTHLLMV
  • DISTINCTCOUNTMV
  • DISTINCTCOUNTRAWHLL
  • DISTINCTCOUNTRAWHLLMV
  • DISTINCTCOUNTRAWTHETASKETCH
  • DISTINCTCOUNTTHETASKETCH
  • DIV
  • DATETIMECONVERT
  • DATETRUNC
  • exp
  • FLOOR
  • FromDateTime
  • FromEpoch
  • FromEpochBucket
  • Histogram
  • hour
  • JSONFORMAT
  • JSONPATH
  • JSONPATHARRAY
  • JSONPATHARRAYDEFAULTEMPTY
  • JSONPATHDOUBLE
  • JSONPATHLONG
  • JSONPATHSTRING
  • jsonextractkey
  • jsonextractscalar
  • length
  • ln
  • lower
  • lpad
  • ltrim
  • max
  • MAXMV
  • MD5
  • millisecond
  • min
  • minmaxrange
  • MINMAXRANGEMV
  • MINMV
  • minute
  • MOD
  • mode
  • month
  • mult
  • now
  • percentile
  • percentileest
  • percentileestmv
  • percentilemv
  • percentiletdigest
  • percentiletdigestmv
  • quarter
  • regexpExtract
  • regexpReplace
  • remove
  • replace
  • reverse
  • round
  • rpad
  • rtrim
  • second
  • SEGMENTPARTITIONEDDISTINCTCOUNT
  • sha
  • sha256
  • sha512
  • sqrt
  • startswith
  • ST_AsBinary
  • ST_AsText
  • ST_Contains
  • ST_Distance
  • ST_GeogFromText
  • ST_GeogFromWKB
  • ST_GeometryType
  • ST_GeomFromText
  • ST_GeomFromWKB
  • STPOINT
  • ST_Polygon
  • strpos
  • ST_Union
  • SUB
  • substr
  • sum
  • summv
  • TIMECONVERT
  • timezoneHour
  • timezoneMinute
  • ToDateTime
  • ToEpoch
  • ToEpochBucket
  • ToEpochRounded
  • TOJSONMAPSTR
  • toGeometry
  • toSphericalGeography
  • trim
  • upper
  • Url
  • UTF8
  • VALUEIN
  • week
  • year
  • yearOfWeek

RESOURCES

• Community
• Team
• Blogs
• Presentations
• Videos

Integrations

• Tableau
• Trino
• ThirdEye
• Superset
• Presto