Files
authentik/website/docs/installation/storage-s3.md
Marc 'risson' Schmitt abc0c2d2a2 root: Multi-tenancy (#7590)
* tenants -> brands, init new tenant model, migrate some config to tenants

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* setup logging for tenants

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* configure celery and cache

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* small fixes, runs

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* task fixes, creation of tenant now works by cloning a template schema, some other small stuff

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* lint

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* fix-tests

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* upstream fixes

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* fix-pylint

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* lint

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* fix tests

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* fix avatar tests

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* migrate config reputation_expiry as well

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* fix web rebase

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* lint

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* fix migrations for template schema

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* fix migrations for template schema

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* fix migrations for template schema 3

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* revert reputation expiry migration

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* fix type

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* fix some more tests

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* website: tenants -> brands

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* try fixing e2e tests

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* start frontend :help:

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* add ability to disable tenants api

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* delete embedded outpost if it is disabled

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* make sure embedded outpost is disabled when tenants are enabled

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* management commands: add --schema option where relevant

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* store files per-tenant

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* fix embedded outpost deletion

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* lint

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* fix files migration

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* add tenant api tests

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* add domain tests

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* add settings tests

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* make --schema-name default to public in mgmt commands

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* lint

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* sources/ldap: make sure lock is per-tenant

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* fix stuff I broke

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* fix remaining failing tests

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* lint

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* try fixing e2e tests

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* much better frontend, but save does not refresh form properly

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* update django-tenants with latest fixes

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* lint

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* i18n-extract

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* review comments

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* move event_retention from brands to tenants

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* wip

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* root: add support for storing media files in S3

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* use permissions for settings api

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* lint

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* blueprints: disable tenants management

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* fix tests

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* fix embedded outpost create/delete logic

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* make gen

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* make sure prometheus metrics are correctly served

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* makefile: don't delete the go api client when not regenerating it

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* tenants api: add recovery group and token creation endpoints

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* fix startup

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* fix prometheus metrics

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* fix tests

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* lint

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* fix web stuff

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* fix migrations from stable

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* fix oauth source type import

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* Revert "fix oauth source type import"

This reverts commit d015fd0244.

* try with setting_changed signal

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* try with connection_created signal

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* fix scim tests

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* fix web after merge

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* fix enterprise settings

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* Revert "try with connection_created signal"

This reverts commit 764a999db8.

* Revert "try with setting_changed signal"

This reverts commit 32b40a3bbb.

* lib/expression: refactor expression compilation

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* fix django version

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* fix web after merge

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* relock poetry

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* fix reconcile

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* try running tenant save in a transaction

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* black

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* test: export postgres logs for debugging and use failfast

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* test: fix container name for logs

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* do not copy tenant data

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* Revert "try running tenant save in a transaction"

This reverts commit da6dec5a61.

* Revert "do not copy tenant data"

This reverts commit d07ae9423672f068b0bd8be409ff9b58452a80f2.

* Revert "Revert "do not copy tenant data""

This reverts commit 4bffb19704.

* fix clone with nodata

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* why not

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* remove failfast

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* remove postgres query logging

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* update reconcile logic to clearly differentiate between tenant and global

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* fix

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* fix reconcile app decorator

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* enable django checks

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* actually nodata was unnecessary as we're cloning from template and not from public

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* pylint

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* update django-tenants with sequence fix

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* actually update

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* fix e2e tests

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* add tests for settings api

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* add tests for recovery api

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* lint

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* recovery tests: do them on a new tenant

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* web: fix system status being degraded when embedded outpost is disabled

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* fix recovery tests

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* fix tenants tests

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* lint-fix

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* lint-fix

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* update UI

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* add management command to create a tenant

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* add docs

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* release notes

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* more docs

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* checklist

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* self review

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* spelling

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* make web after upgrading

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* remove extra xlif file

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* prettier

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* Revert "add management command to create a tenant"

This reverts commit 39d13c0447.

* split api into smaller files, only import urls when tenants is enabled

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* rewite some things on the release notes

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* root: make sure install_id comes from public schema

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* require a license to use tenants

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* lint

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* fix tenants tests

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* fix files migration

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* release notes: add warning about user sessions being invalidated

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

* remove api disabled test, we can't test for it

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>

---------

Signed-off-by: Marc 'risson' Schmitt <marc.schmitt@risson.space>
Signed-off-by: Jens Langhammer <jens@goauthentik.io>
Co-authored-by: Jens Langhammer <jens@goauthentik.io>
2024-01-23 14:28:06 +01:00

3.9 KiB

title
title
S3 storage setup

Preparation

First, create a user on your S3 storage provider and get access credentials for S3, hereafter referred as access_key and secret_key.

You'll also need to know which endpoint authentik is going to use to access the S3 API, hereafter referred as https://s3.provider.

The bucket in which authentik is going to store files is going to be called authentik-media. You may need to change this name depending on your S3 provider limitations. Also, we're suffixing the bucket name with -media as authentik currently only stores media files, but may use other buckets in the future.

The domain used to access authentik is going to be referred to as authentik.company.

You will also need the AWS CLI.

S3 configuration

Bucket creation

Let's create the bucket in which authentik is going to store files:

AWS_ACCESS_KEY_ID=access_key AWS_SECRET_ACCESS_KEY=secret_key aws s3api --endpoint-url=https://s3.provider create-bucket --bucket=authentik-media --acl=private

If using AWS S3, you can omit the --endpoint-url option, but may need to specify the --region option. If using Google Cloud Storage, refer to its documentation on how to create buckets.

The bucket ACL is set to private, although that is not strictly necessary, as an ACL associated with each object stored in the bucket will be private as well.

CORS policy

Next, let's associate a CORS policy to the bucket, to allow the authentik web interface to show images stored in the bucket.

First, save the following file locally as cors.json:

{
    "CORSRules": [
        {
            "AllowedOrigins": ["authentik.company"],
            "AllowedHeaders": ["Authorization"],
            "AllowedMethods": ["GET"],
            "MaxAgeSeconds": 3000
        }
    ]
}

If authentik is accessed from multiple domains, you can add them to the AllowedOrigins list.

Let's apply that policy to the bucket:

AWS_ACCESS_KEY_ID=access_key AWS_SECRET_ACCESS_KEY=secret_key aws s3api --endpoint-url=https://s3.provider put-bucket-cors --bucket=authentik-media --cors-configuration=file://cors.json

Configuring authentik

Add the following to your .env file:

AUTHENTIK_STORAGE_MEDIA_BACKEND=s3
AUTHENTIK_STORAGE_MEDIA_S3_ACCESS__KEY=access_key
AUTHENTIK_STORAGE_MEDIA_S3_SECRET__KEY=secret_key
AUTHENTIK_STORAGE_MEDIA_S3_BUCKET__NAME=authentik-media

If you're using AWS S3 as your S3 provider, add the following:

AUTHENTIK_STORAGE_MEDIA_S3_REGION=us-east-1  # Use the region of the bucket

If you're not using AWS S3 as your S3 provider, add the following:

AUTHENTIK_STORAGE_MEDIA_S3_ENDPOINT=https://s3.provider
AUTHENTIK_STORAGE_MEDIA_S3_CUSTOM__DOMAIN=s3.provider/authentik-media

The ENDPOINT setting specifies how authentik talks to the S3 provider.

The CUSTOM__DOMAIN setting specifies how URLs are constructed to be shown on the web interface. For example, an object stored at application-icons/application.png with a CUSTOM__DOMAIN setting of s3.provider/authentik-media will result in a URL of https://s3.provider/authentik-media/application-icons/application.png. You can also use subdomains for your buckets depending on what your S3 provider offers: authentik-media.s3.provider. Whether HTTPS is used is controlled by the AUTHENTIK_STORAGE_MEDIA_S3_SECURE__URLS which defaults to true.

For more control over settings, refer to the configuration reference

Migrating between storage backends

The following section assumes that the local storage path is /media and the bucket name is authentik-media. It also assumes you have a working aws CLI that can interact with the bucket.

From file to s3

Follow the setup steps above, and then migrate the files from your local directory to s3:

aws s3 sync /media s3://authentik-media

From s3 to file

aws s3 sync s3://authentik-media /media