Add basic auth (#14)

* Implement HTTP auth; fix typos and formatting

* changelog

CHANGELOG.md

@@ -6,6 +6,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.0.14] - 2020-05-30
+
+### Added
+
+- HTTP Basic Authentication
+
 ## [0.0.13] - 2020-05-30
 
 ### Added

README.md

@@ -17,6 +17,7 @@ Git as Terraform backend? Seriously? I know, might sound like a stupid idea at f
     - [Git Credentials](#git-credentials)
     - [State Encryption](#state-encryption)
     - [Running backend remotely](#running-backend-remotely)
+    - [Basic HTTP Authentication](#basic-http-authentication)
     - [Why not native Terraform Backend](#why-not-native-terraform-backend)
   - [Why storing state in Git](#why-storing-state-in-git)
   - [Proposed solution](#proposed-solution)
@@ -131,14 +132,14 @@ This is so we could have more Storage Types supported in the future as well as m
 
 ### Configuration
 
-CLI | `terraform-backend-git.hcl` | HTTP | Description
+CLI | `terraform-backend-git.hcl` | Environment Variable | TF HTTP backend config | Description
 --- | --- | --- | ---
-`--repository` | `git.repository` | `repository` | Required; Which repository to use for storing TF state?
-`--ref` | `git.ref` | `ref` | Optional; Which branch to use in that `repository`? Default: `master`.
-`--state` | `git.state` | `state` | Required; Path to the state file in that `repository`.
-`--config` | - | - | Optional; Path to the `hcl` config file.
-`--address` | `address` | - | Optional; Local binding address and port to listen for HTTP requests. Only change the port, **do not change the address to `0.0.0.0` before you read [Running backend remotely](#running-backend-remotely)**. Default: `127.0.0.1:6061`.
-`--access-logs` | `accessLogs` | - | Optional; Set to `true` to enable HTTP access logs on backend. Default: `false`.
+`--repository` | `git.repository` | `TF_BACKEND_GIT_GIT_REPOSITORY` |`repository` | Required; Which repository to use for storing TF state?
+`--ref` | `git.ref` | `TF_BACKEND_GIT_GIT_REF` |`ref` | Optional; Which branch to use in that `repository`? Default: `master`.
+`--state` | `git.state` | `TF_BACKEND_GIT_GIT_STATE` | `state` | Required; Path to the state file in that `repository`.
+`--config` | - | - | - | Optional; Path to the `hcl` config file.
+`--address` | `address` | `TF_BACKEND_GIT_ADDRESS` | - | Optional; Local binding address and port to listen for HTTP requests. Only change the port, **do not change the address to `0.0.0.0` before you read [Running backend remotely](#running-backend-remotely)**. Default: `127.0.0.1:6061`.
+`--access-logs` | `accessLogs` | `TF_BACKEND_GIT_ACCESSLOGS` | - | Optional; Set to `true` to enable HTTP access logs on backend. Default: `false`.
 
 ### Git Credentials
 
@@ -168,11 +169,33 @@ First of all, **DON'T DO IT**.
 
 It can be done, but again - **DON'T DO IT**.
 
-Besides the fact that Terraform does not perform any encryption before sending the state to HTTP backend, there is also **no authentication whatsoever**. Running remotely accessible backend like this would **not** be secure - **anyone who can make HTTP calls to it would be able to get, update or delete your state files with no credentials**.
+First of all, by default, Terraform does not perform any encryption before sending the state to HTTP backend. Also, running remotely accessible backend like this without authentication would **not** be secure - **anyone who can make HTTP calls to it would be able to get, update or delete your state files**.
 
-Make sure you do not open the port in your firewall for remote connections. By default it would start on port `6061` and would use `127.0.0.1` as the binding address, so that nothing would be able to connect remotely. That would still not protect you from local loop interface traffic interception or spoofing (or even having a bad actor who already got the access to the host to send HTTP requests directly to the endpoint), but that's the best we can do for now, either until this implementation gets into Terraform as a native Backend implementation, or Backends become a pluggable options, or gRCP backend being implemented or Terraform adds some auth/encryption options to the HTTP backend protocol, or some other miracle.
+But even then, this backend is not aiming to become a standalone project. Once backends in Terraform [can be pluggable gRPC components](https://github.com/hashicorp/terraform/issues/5877), this backend will be converted to a normal TF gRPC plugin, HTTP support will be removed, and binaries will not be distributed separately anymore (I believe TF will be able to fetch them automatically just like it does it for providers right now). Until that happens, basically HTTP protocol is used instead of gRPC, and downloading and running this backend is delegated to the user. Therefore this backend recommended to be used in plugin/wrapper notion, i.e. you start it just before running Terraform and then you stop it right after Terraform is finished, and it happens on the same host. The `wrapper` mode makes that very scenario even easier, it run Terraform for you so you don't have to maintain multiple console windows. At the end of the day, you are not running Terraform AWS Provider remotely, are you?
 
-You may get creative and use something like K8s Network Policies like `calico`, or wrap backend traffic into API Gateway or ServiceMesh like Istio to add encryption and authentication, and then and only then you will want to use option `--address=:6061` so the backend will bind to `0.0.0.0` and become remotely accessible.
+Even though the traffic can be secured with HTTP TLS encryption ([WIP](https://github.com/plumber-cd/terraform-backend-git/issues/12)), and [Basic HTTP Authentication](#basic-http-authentication) can be added, authentication and encryption is there just for the sake of securing local traffic, and even when it's enabled - remote operations mode is not recommended.
+
+Therefore it will not be considered to implement any rich HTTP-related features such as AD/Okta HTTP authentication, or any other features that will move this project further away from the goal to become a gRPC plugin.
+
+Make sure you do not open the port in your firewall for remote connections. By default it would start on port `6061` and would use `127.0.0.1` as the binding address, so that nothing would be able to connect remotely. That would still not protect you from local loop interface traffic interception or spoofing (or even having a bad actor who already got the access to the host to send HTTP requests directly to the endpoint), so consider enabling Basic HTTP Authentication and TLS encryption.
+
+You may get creative and use something like K8s Network Policies like `calico`, or wrap backend traffic into API Gateway or ServiceMesh like Istio to add external layer of encryption and authentication, and then at your discretion you may run it with `--address=:6061` argument so the backend will bind to `0.0.0.0` and become remotely accessible.
+
+### Basic HTTP Authentication
+
+You can use `TF_BACKEND_GIT_HTTP_USERNAME` and `TF_BACKEND_GIT_HTTP_PASSWORD` environment variables to add an extra layer of protection. In `wrapper` mode, same environment variables will be used to render `*.auto.tf` config for Terraform, but if you are using backend in standalone mode - you will have to tell these credentials to the Terraform explicitly:
+
+```terraform
+terraform {
+  backend "http" {
+    ...
+    username = "user"
+    password = "pswd"
+  }
+}
+```
+
+Note that if either username or password changes, Terraform will consider this as a backend configuration change and will want to ask you to migrate state. Since backend will not be accepting old credentials anymore - it will fail to `init` (can't read the "old" state). Consider deleting your local `.terraform/terraform.tfstate` file to fix this.
 
 ### Why not native Terraform Backend
 

git_backend_wrappers.go

@@ -8,6 +8,8 @@ import (
 
 	"github.com/spf13/cobra"
 	"github.com/spf13/viper"
+
+	"github.com/plumber-cd/terraform-backend-git/server"
 )
 
 // gitHTTPBackendConfigPath is a path to the backend tf config to generate
@@ -25,21 +27,28 @@ terraform {
 		address = "http://localhost:{{ .port }}/?type=git&repository={{ .repository }}&ref={{ .ref }}&state={{ .state }}"
 		lock_address = "http://localhost:{{ .port }}/?type=git&repository={{ .repository }}&ref={{ .ref }}&state={{ .state }}"
 		unlock_address = "http://localhost:{{ .port }}/?type=git&repository={{ .repository }}&ref={{ .ref }}&state={{ .state }}"
+		username = "{{ .username }}"
+		password = "{{ .password }}"
 	}
 }
 		`)
 		if err != nil {
 			log.Fatal(err)
 		}
 
+		username, _ := os.LookupEnv("TF_BACKEND_GIT_HTTP_USERNAME")
+		password, _ := os.LookupEnv("TF_BACKEND_GIT_HTTP_PASSWORD")
+
 		addr := strings.Split(viper.GetString("address"), ":")
 		p := map[string]string{
-			"port": addr[len(addr)-1],
+			"port":     addr[len(addr)-1],
+			"username": username,
+			"password": password,
 		}
 
 		for _, flag := range []string{"repository", "ref", "state"} {
 			if p[flag] = viper.GetString("git." + flag); p[flag] == "" {
-				log.Fatal(err)
+				log.Fatalf("%s must be set", flag)
 			}
 		}
 
@@ -53,7 +62,7 @@ terraform {
 			log.Fatal(err)
 		}
 
-		go startServer()
+		go server.Start()
 	},
 	PersistentPostRun: func(cmd *cobra.Command, args []string) {
 		if err := os.Remove(gitHTTPBackendConfigPath); err != nil {

main.go

@@ -3,12 +3,10 @@ package main
 import (
 	"fmt"
 	"log"
-	"net/http"
 	"os"
 	"os/exec"
 	"strings"
 
-	"github.com/gorilla/handlers"
 	"github.com/mitchellh/go-homedir"
 	"github.com/plumber-cd/terraform-backend-git/backend"
 	"github.com/plumber-cd/terraform-backend-git/server"
@@ -18,6 +16,7 @@ import (
 	"github.com/spf13/viper"
 )
 
+// Version holds the version binary built with - must be injected from build process via -ldflags="-X 'main.Version=vX.Y.Z'"
 var Version = "development"
 
 var cfgFile string
@@ -32,7 +31,7 @@ var rootCmd = &cobra.Command{
 			log.Fatal(err)
 		}
 
-		startServer()
+		server.Start()
 	},
 }
 
@@ -68,26 +67,6 @@ var wrappersCmds = []*cobra.Command{
 	terraformWrapperCmd,
 }
 
-// startServer listen for traffic
-func startServer() {
-	backend.KnownStorageTypes = map[string]types.StorageClient{
-		"git": git.NewStorageClient(),
-	}
-
-	http.HandleFunc("/", server.HandleFunc)
-
-	var handler http.Handler
-	if viper.GetBool("accessLogs") {
-		handler = handlers.LoggingHandler(os.Stdout, http.DefaultServeMux)
-	} else {
-		handler = nil
-	}
-
-	address := viper.GetString("address")
-	log.Println("listen on", address)
-	log.Fatal(http.ListenAndServe(address, handler))
-}
-
 func initConfig() {
 	viper.SetConfigType("hcl")
 	viper.SetConfigName("terraform-backend-git")
@@ -118,6 +97,10 @@ func initConfig() {
 }
 
 func main() {
+	backend.KnownStorageTypes = map[string]types.StorageClient{
+		"git": git.NewStorageClient(),
+	}
+
 	// keep the output clean as in wrapper mode it'll mess out with Terraform own output
 	log.SetFlags(0)
 	log.SetPrefix("[terraform-backend-git]: ")

server/server.go

@@ -2,17 +2,97 @@
 package server
 
 import (
+	"crypto/subtle"
 	"errors"
 	"io/ioutil"
 	"log"
 	"net/http"
+	"os"
 
+	"github.com/gorilla/handlers"
 	"github.com/plumber-cd/terraform-backend-git/backend"
+	"github.com/plumber-cd/terraform-backend-git/crypt"
 	"github.com/plumber-cd/terraform-backend-git/types"
+	"github.com/spf13/viper"
 )
 
-// HandleFunc main function responsible for routing
-func HandleFunc(response http.ResponseWriter, request *http.Request) {
+// Start listen for traffic
+func Start() {
+	var h http.Handler
+
+	h = http.HandlerFunc(handleFunc)
+
+	h = basicAuth(h)
+
+	if viper.GetBool("accessLogs") {
+		log.Println("WARNING: Access Logs enabled")
+		h = handlers.LoggingHandler(os.Stdout, h)
+	}
+
+	mux := http.NewServeMux()
+	mux.Handle("/", h)
+
+	address := viper.GetString("address")
+	log.Println("listen on", address)
+	log.Fatal(http.ListenAndServe(address, mux))
+}
+
+// basicAuth checking for user authentication
+func basicAuth(next http.Handler) http.Handler {
+	backendUsername, okBackendUsername := os.LookupEnv("TF_BACKEND_GIT_HTTP_USERNAME")
+	backendPassword, okBackendPassword := os.LookupEnv("TF_BACKEND_GIT_HTTP_PASSWORD")
+	if !okBackendUsername || !okBackendPassword {
+		log.Println("WARNING: HTTP basic auth is disabled, please specify TF_BACKEND_GIT_HTTP_USERNAME and TF_BACKEND_GIT_HTTP_PASSWORD")
+		return next
+	}
+
+	backendUsername, err := crypt.MD5(backendUsername)
+	if err != nil {
+		log.Fatal(err)
+	}
+
+	backendPassword, err = crypt.MD5(backendPassword)
+	if err != nil {
+		log.Fatal(err)
+	}
+
+	return http.HandlerFunc(func(response http.ResponseWriter, request *http.Request) {
+		handler := handler{
+			Request:  request,
+			Response: response,
+		}
+
+		u, p, ok := request.BasicAuth()
+		if !ok {
+			handler.serverError(types.ErrUnauthorized)
+			return
+		}
+
+		u, err := crypt.MD5(u)
+		if err != nil {
+			handler.serverError(types.ErrUnauthorized)
+			return
+		}
+
+		p, err = crypt.MD5(p)
+		if err != nil {
+			handler.serverError(types.ErrUnauthorized)
+			return
+		}
+
+		if subtle.ConstantTimeCompare([]byte(u), []byte(backendUsername)) != 1 || subtle.ConstantTimeCompare([]byte(p), []byte(backendPassword)) != 1 {
+			handler.clientError(types.ErrUnauthorized)
+			return
+		}
+
+		if next != nil {
+			next.ServeHTTP(response, request)
+		}
+	})
+}
+
+// handleFunc main function responsible for routing
+func handleFunc(response http.ResponseWriter, request *http.Request) {
 	handler := handler{
 		Request:  request,
 		Response: response,
@@ -137,7 +217,7 @@ func (handler *handler) clientError(err error) {
 	handler.responseError(http.StatusBadRequest, "400 - Bad Request", err)
 }
 
-// responseError is a handler that will try to read known errors and formulate approapriate responses to them
+// responseError is a handler that will try to read known errors and formulate appropriate responses to them
 // If error was unknown, just use defaultCode and defaultResponse error message.
 func (handler *handler) responseError(defaultCode int, defaultResponse string, actualErr error) {
 	log.Printf("%s", actualErr)
@@ -152,9 +232,14 @@ func (handler *handler) responseError(defaultCode int, defaultResponse string, a
 			handler.Response.Write([]byte("428 - Locking Required"))
 		case types.ErrStateDidNotExisted:
 			handler.Response.WriteHeader(http.StatusNoContent)
+		case types.ErrUnauthorized:
+			handler.Response.Header().Set("WWW-Authenticate", `Basic realm=terraform-backend-git`)
+			handler.Response.WriteHeader(http.StatusUnauthorized)
+			handler.Response.Write([]byte("401 - Unauthorized"))
 		default:
 			handler.Response.WriteHeader(defaultCode)
 			handler.Response.Write([]byte(defaultResponse))
 		}
+
 	}
 }

test/tf/terraform-backend-git.hcl

@@ -1,2 +1,2 @@
 git.repository = "git@github.com:plumber-cd/terraform-backend-git-fixture-state.git"
-git.state = "state.json"
+git.state = "state.json"

test/tf2/main.tf

@@ -3,6 +3,8 @@ terraform {
     address = "http://localhost:6061/?type=git&repository=git@github.com:plumber-cd/terraform-backend-git-fixture-state.git&ref=master&state=state2.json"
     lock_address = "http://localhost:6061/?type=git&repository=git@github.com:plumber-cd/terraform-backend-git-fixture-state.git&ref=master&state=state2.json"
     unlock_address = "http://localhost:6061/?type=git&repository=git@github.com:plumber-cd/terraform-backend-git-fixture-state.git&ref=master&state=state2.json"
+    username = "user"
+    password = "1234"
   }
 }
 

types/types.go

@@ -21,10 +21,12 @@ func (err *ErrLocked) Error() string {
 var (
 	// ErrStateDidNotExisted indicate that the state did not existed
 	ErrStateDidNotExisted = errors.New("state did not existed")
-	// ErrLockingConflict indicate the lock was already aquired by someone else
-	ErrLockingConflict = errors.New("the lock was already aquired by someone else")
+	// ErrLockingConflict indicate the lock was already acquired by someone else
+	ErrLockingConflict = errors.New("the lock was already acquired by someone else")
 	// ErrLockMissing indicate that the lock didn't existed when it was expected/required to
 	ErrLockMissing = errors.New("was not locked")
+	// ErrUnauthorized indicates that the action was not authorized
+	ErrUnauthorized = errors.New("Unauthorized")
 )
 
 // LockInfo represents a TF Lock Metadata.