Move lasso SQL cache in Steve

go.mod

@@ -44,6 +44,8 @@ require (
 	k8s.io/kube-aggregator v0.31.1
 	k8s.io/kube-openapi v0.0.0-20240411171206-dc4e619f62f3
 	k8s.io/kubernetes v1.31.1
+	k8s.io/utils v0.0.0-20240711033017-18e509b52bc8
+	modernc.org/sqlite v1.29.10
 	sigs.k8s.io/controller-runtime v0.19.0
 )
 
@@ -137,12 +139,10 @@ require (
 	k8s.io/component-base v0.31.1 // indirect
 	k8s.io/klog/v2 v2.130.1 // indirect
 	k8s.io/kms v0.31.1 // indirect
-	k8s.io/utils v0.0.0-20240711033017-18e509b52bc8 // indirect
 	modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6 // indirect
 	modernc.org/libc v1.49.3 // indirect
 	modernc.org/mathutil v1.6.0 // indirect
 	modernc.org/memory v1.8.0 // indirect
-	modernc.org/sqlite v1.29.10 // indirect
 	modernc.org/strutil v1.2.0 // indirect
 	modernc.org/token v1.1.0 // indirect
 	sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.30.3 // indirect

pkg/server/server.go

@@ -82,7 +82,7 @@ type Options struct {
 	AggregationSecretName      string
 	ClusterRegistry            string
 	ServerVersion              string
-	// SQLCache enables the SQLite-based lasso caching mechanism
+	// SQLCache enables the SQLite-based caching mechanism
 	SQLCache bool
 
 	// ExtensionAPIServer enables an extension API server that will be served

pkg/sqlcache/Readme.md

@@ -0,0 +1,157 @@
+# SQL Cache
+
+## Sections
+- [ListOptions Informer](#listoptions-informer)
+  - [List Options](#list-options)
+  - [ListOption Indexer](#listoptions-indexer)
+    - [SQL Store](#sql-store)
+    - [Partitions](#partitions)
+- [How to Use](#how-to-use)
+- [Technical Information](#technical-information)
+  - [SQL Tables](#sql-tables)
+  - [SQLite Driver](#sqlite-driver)
+  - [Connection Pooling](#connection-pooling)
+  - [Encryption Defaults](#encryption-defaults)
+  - [Indexed Fields](#indexed-fields)
+  - [ListOptions Behavior](#listoptions-behavior)
+  - [Troubleshooting Sqlite](#troubleshooting-sqlite)
+
+
+
+## ListOptions Informer
+The main usable feature from the SQL cache is the ListOptions Informer. The ListOptionsInformer provides listing functionality,
+like any other informer, but with a wider array of options. The options are configured by informer.ListOptions.
+
+### List Options
+ListOptions includes the following:
+* Match filters for indexed fields. Filters are for specifying the value a given field in an object should be in order to
+be included in the list. Filters can be set to equals or not equals. Filters can be set to look for partial matches or
+exact (strict) matches. Filters can be OR'd and AND'd with one another. Filters only work on fields that have been indexed.
+* Primary field and secondary field sorting order. Can choose up to two fields to sort on. Sort order can be ascending
+or descending. Default sorting is to sort on metadata.namespace in ascending first and then sort on metadata.name.
+* Page size to specify how many items to include in a response.
+* Page number to specify offset. For example, a page size of 50 and a page number of 2, will return items starting at
+index 50. Index will be dependent on sort. Page numbers start at 1.
+
+### ListOptions Factory
+The ListOptions Factory helps manage multiple ListOption Informers. A user can call Factory.InformerFor(), to create new
+ListOptions informers if they do not exist and retrieve existing ones.
+
+### ListOptions Indexer
+Like all other informers, the ListOptions informer uses an indexer to cache objects of the informer's type. A few features
+set the ListOptions Indexer apart from others indexers:
+* an on-disk store instead of an in-memory store.
+* accepts list options backed by SQL queries for extended search/filter/sorting capability.
+* AES GCM encryption using key hierarchy.
+
+### SQL Store
+The SQL store is the main interface for interacting with the database. This store backs the indexer, and provides all
+functionality required by the cache.Store interface.
+
+### Partitions
+Partitions are constraints for ListOptionsInform ListByOptions() method that are separate from ListOptions. Partitions
+are strict conditions that dictate which namespaces or names can be searched from. These overrule ListOptions and are
+intended to be used as a way of enforcing RBAC.
+
+## How to Use
+```go
+    package main
+    import(
+		"k8s.io/client-go/dynamic"
+        "github.com/rancher/steve/pkg/sqlcache/informer"
+		"github.com/rancher/steve/pkg/sqlcache/informer/factory"
+    )
+
+    func main() {
+		cacheFactory, err := factory.NewCacheFactory()
+		if err != nil {
+			panic(err)
+        }
+		// config should be some rest config created from kubeconfig
+		// there are other ways to create a config and any client that conforms to k8s.io/client-go/dynamic.ResourceInterface
+		// will work.
+		client, err := dynamic.NewForConfig(config)
+		if err != nil {
+			panic(err)
+		}
+
+		fields := [][]string{{"metadata", "name"}, {"metadata", "namespace"}}
+		opts := &informer.ListOptions{}
+		// gvk should be of type k8s.io/apimachinery/pkg/runtime/schema.GroupVersionKind
+		c, err := cacheFactory.CacheFor(fields, client, gvk)
+		if err != nil {
+			panic(err)
+		}
+
+		// continueToken will just be an offset that can be used in Resume on a subsequent request to continue
+		// to next page
+		list, continueToken, err := c.ListByOptions(apiOp.Context(), opts, partitions, namespace)
+		if err != nil {
+			panic(err)
+		}
+	}
+```
+
+## Technical Information
+
+### SQL Tables
+There are three tables that are created for the ListOption informer:
+* object table - this contains objects, including all their fields, as blobs. These blobs may be encrypted.
+* fields table - this contains specific fields of value for objects. These are specified on informer create and are fields
+that it is desired to filter or order on.
+* indices table - the indices table stores indexes created and objects' values for each index. This backs the generic indexer
+that contains the functionality needed to conform to cache.Indexer.
+
+### SQLite Driver
+There are multiple SQLite drivers that this package could have used. One of the most, if not the most, popular SQLite golang
+drivers is [mattn/go-sqlite3](https://github.com/mattn/go-sqlite3). This driver is not being used because it requires enabling
+the cgo option when compiling and at the moment steve's main consumer, rancher, does not compile with cgo. We did not want
+the SQL informer to be the sole driver in switching to using cgo. Instead, modernc's driver which is in pure golang. Side-by-side
+comparisons can be found indicating the cgo version is, as expected, more performant. If in the future it is deemed worthwhile
+then the driver can be easily switched by replacing the empty import in `pkg/cache/sql/store` from `_ "modernc.org/sqlite"` to `_ "github.com/mattn/go-sqlite3"`.
+
+### Connection Pooling
+While working with the `database/sql` package for go, it is important to understand how sql.Open() and other methods manage
+connections. Open starts a connection pool; that is to say after calling open once, there may be anywhere from zero to many
+connections attached to a sql.Connection. `database/sql` manages this connection pool under the hood. In most cases, an
+application only need one sql.Connection, although sometimes application use two: one for writes, the other for reads. To
+read more about the `sql` package's connection pooling read [Managing connections](https://go.dev/doc/database/manage-connections).
+
+The use of connection pooling and the fact that steve potentially has many go routines accessing the same connection pool,
+means we have to be careful with writes. Exclusively using sql transaction to write helps ensure safety. To read more about
+sql transactions read SQLite's [Transaction docs](https://www.sqlite.org/lang_transaction.html).
+
+### Encryption Defaults
+By default only specified types are encrypted. These types are hard-coded and defined by defaultEncryptedResourceTypes
+in `pkg/cache/sql/informer/factory/informer_factory.go`. To enabled encryption for all types, set the ENV variable
+`CATTLE_ENCRYPT_CACHE_ALL` to "true".
+
+The key size used is 256 bits. Data-encryption-keys are stored in the object table and are rotated every 150,000 writes.
+
+### Indexed Fields
+Filtering and sorting only work on indexed fields. These fields are defined when using `CacheFor`. Objects will
+have the following indexes by default:
+* Fields in informer.defaultIndexedFields
+* Fields passed to InformerFor()
+
+### ListOptions Behavior
+Defaults:
+* Sort.PrimaryField: `metadata.namespace`
+* Sort.SecondaryField: `metadata.name`
+* Sort.PrimaryOrder: `ASC` (ascending)
+* Sort.SecondaryOrder: `ASC` (ascending)
+* All filters have partial matching set to false by default
+
+There are some uncommon ways someone could use ListOptions where it would be difficult to predict what the result would be.
+Below is a non-exhaustive list of some of these cases and what the behavior is:
+* Setting Pagination.Page but not Pagination.PageSize will cause Page to be ignored
+* Setting Sort.SecondaryField only will sort as though it was Sort.PrimaryField. Sort.SecondaryOrder will still be applied
+and Sort.PrimaryOrder will be ignored
+
+### Writing Secure Queries
+Values should be supplied to SQL queries using placeholders, read [Avoiding SQL Injection Risk](https://go.dev/doc/database/sql-injection). Any other portions
+of a query that may be user supplied, such as columns, should be carefully validated against a fixed set of acceptable values.
+
+### Troubleshooting SQLite
+A useful tool for troubleshooting the database files is the sqlite command line tool. Another useful tool is the goland
+sqlite plugin. Both of these tools can be used with the database files.