habsi/authentik
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

web: Replace lingui.js with lit-localize (#5761)

Browse Source 
* \#\# Details

web: replace lingui with lit/localize

\#\# Changes

This rather massive shift replaces the lingui and `t()` syntax with lit-localize, XLIFF, and the `msg()`
syntax used by lit-localize.  90% of this work was mechanized; simple perl scripts found and replaced
all uses of `t()` with the appropriate corresponding syntax for `msg()` and `msg(str())`.

The XLIFF files were auto-generated from the PO files.  They have not been audited, and they should be
checked over by professional translators.  The actual _strings_ have not been changed, but as this was
a mechanized change there is always the possibility of mis-translation-- not by the translator, but by
the script.

* web: revise lit/localize: fix two installation issues.

* web: revise localization

TL;DR:

- Replaced all of Lingui's `t()` syntax with `msg()` syntax.
- Mechanically (i.e with a script) converted all of the PO files to XLIFF files
- Refactored the localization code to be a bit smarter:
  - the function `getBestMatchLocale` takes the locale lists and a requested locale, and returns the
    first match of:
    - The locale's code exactly matches the requested locale
    - The locale code exactly matches the prefix of the requested locale (i.e the "en" part of "en-US")
    - the locale code's prefix exactly matches the prefix of the requested locale
    This function is passed to lit-locate's `loadLocale()`.
  - `activateLocale()` just calls `loadLocale()` now.
  - `autodetectLanguage` searches the following, and picks the first that returns a valid locale
    object, before passing it to `loadLocale()`:
    - The User's settings
    - A `?locale=` component found in `window.location.search`
    - The `window.navigator.language` field
    - English

The `msg()` only runs when it's run.  This seems obvious, but it means that you cannot cache
strings at load time; they must be kept inside functions that are re-run so that the `msg()` engine
can look up the strings in the preferred language of the user at that moment.

You can use thunks-of-strings if you really need them that way.

* Including the 'xliff-converter' in case anyone wants to review it.

* The xliff-converter is tagged as 'xliff-converter', but has been
deleted.

\#\# Details

-   Resolves #5171

\#\# Changes

\#\#\# New Features

-   Adds a "Add an Application" to the LibraryView if there are no applications and the user is an administrator.

\#\#\# Breaking Changes

-   Adds breaking change which causes \<issue\>.

\#\# Checklist

-   [ ] Local tests pass (`ak test authentik/`)
-   [ ] The code has been formatted (`make lint-fix`)

If an API change has been made

-   [ ] The API schema has been updated (`make gen-build`)

If changes to the frontend have been made

-   [ ] The code has been formatted (`make web`)
-   [ ] The translation files have been updated (`make i18n-extract`)

If applicable

-   [ ] The documentation has been updated
-   [ ] The documentation has been formatted (`make website`)

* web: fix redundant locales for zh suite.

* web: prettier pass for locale update

* web: localization moderization

Changed the names of the lit-localize commands to make it clear they're
part of the localization effort, and not just "build" and "extract".

* update transifex config

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* fix package lock?

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* use build not compile

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* web: conversion to lit-localize

The CI produced a list of problems that I hadn't caught earlier,
due to a typo ("localize build" is correct, "localize compile" is
not) I had left in package.json.  They were minor and linty, but
it was still wise to fix them.

* web: replace lingui with lit/locale

This commit fixes some minor linting issues that were hidden by a typo in package.json.  The
issues were not apparently problematic from a Javascript point of view, but they pointed
to sloppy thinking in the progression of types through the system, so I cleaned them
up and formalized the types from LocaleModule to AkLocale.

* web: replace lingui with lit/localize

One problem that has repeatedly come up is that localize's templates do not produce
JavaScript that conforms with our shop style.  I've replaced `build-locale` with
a two-step that builds the locale *and* ensures that it conforms to the shop style
via `prettier` every time.

* web: replace lingui with lit-locale

This commit applies the most recent bundle of translations to the
new lit-locale aspect component.  It also revises the algorithm
for *finding* the correct locale, replacing the complex fall-back
with some rather straightforward regular expressions.

In the case of Chinese, the fallback comes at the end of the
selection list, which may not be, er, politically valuable
(since Taiwan and Hong Kong come before, being exceptions that
need to be tested).  If we need a different order for presentation,
that'll be a future feature.

* web: replace lingui with lit/locale

Well, that was embarassing.

---------

Signed-off-by: Jens Langhammer <jens@goauthentik.io>
Co-authored-by: Jens Langhammer <jens@goauthentik.io>
This commit is contained in:
Ken Sternberg
2023-06-02 08:08:36 -07:00
committed by GitHub
parent afa8a505ee
commit 44a057ed9c
273 changed files with 64104 additions and 117217 deletions
Download Patch File Download Diff File Expand all files Collapse all files

178
web/src/common/ui/locale.ts
View File

@ -1,178 +0,0 @@
import { EVENT_LOCALE_CHANGE } from "@goauthentik/common/constants";
import { globalAK } from "@goauthentik/common/global";
import { messages as enLocale } from "@goauthentik/locales/en";
import { Messages, i18n } from "@lingui/core";
import { fromNavigator, fromUrl } from "@lingui/detect-locale";
import { t } from "@lingui/macro";
interface Locale {
locale: Messages;
}
export const LOCALES: {
code: string;
label: string;
locale: () => Promise<Locale>;
}[] = [
{
code: "en",
label: t`English`,
locale: async () => {
return {
locale: enLocale,
};
},
},
{
code: "debug",
label: t`Debug`,
locale: async () => {
return {
locale: (await import("@goauthentik/locales/pseudo-LOCALE")).messages,
};
},
},
{
code: "fr",
label: t`French`,
locale: async () => {
return {
locale: (await import("@goauthentik/locales/fr_FR")).messages,
};
},
},
{
code: "tr",
label: t`Turkish`,
locale: async () => {
return {
locale: (await import("@goauthentik/locales/tr")).messages,
};
},
},
{
code: "es",
label: t`Spanish`,
locale: async () => {
return {
locale: (await import("@goauthentik/locales/es")).messages,
};
},
},
{
code: "pl",
label: t`Polish`,
locale: async () => {
return {
locale: (await import("@goauthentik/locales/pl")).messages,
};
},
},
{
code: "zh_TW",
label: t`Taiwanese Mandarin`,
locale: async () => {
return {
locale: (await import("@goauthentik/locales/zh_TW")).messages,
};
},
},
{
code: "zh-CN",
label: t`Chinese (simplified)`,
locale: async () => {
return {
locale: (await import("@goauthentik/locales/zh-Hans")).messages,
};
},
},
{
code: "zh-HK",
label: t`Chinese (traditional)`,
locale: async () => {
return {
locale: (await import("@goauthentik/locales/zh-Hant")).messages,
};
},
},
{
code: "de",
label: t`German`,
locale: async () => {
return {
locale: (await import("@goauthentik/locales/de")).messages,
};
},
},
];
const DEFAULT_FALLBACK = () => "en";
export function autoDetectLanguage() {
// Always load en locale at the start so we have something and don't error
i18n.load("en", enLocale);
i18n.activate("en");
const locales: string[] = [];
// Get all locales we can, in order
// - Global authentik settings (contains user settings)
// - URL parameter
// - Navigator
// - Fallback (en)
// Remove any invalid values, add broader locales (fr-FR becomes fr)
// Remove any duplicate values
[globalAK()?.locale || "", fromUrl("locale"), fromNavigator(), DEFAULT_FALLBACK()]
.filter((v) => v && v !== "")
.map((locale) => {
locales.push(locale);
// For now we only care about the first locale part
if (locale.includes("_")) {
locales.push(locale.split("_")[0]);
}
if (locale.includes("-")) {
locales.push(locale.split("-")[0]);
}
})
.filter((v, idx, arr) => {
return arr.indexOf(v) === idx;
});
console.debug(`authentik/local: Locales to try: ${locales}`);
for (const tryLocale of locales) {
if (LOCALES.find((locale) => locale.code === tryLocale)) {
console.debug(`authentik/locale: Activating detected locale '${tryLocale}'`);
activateLocale(tryLocale);
return;
} else {
console.debug(`authentik/locale: No matching locale for ${tryLocale}`);
}
}
console.debug(`authentik/locale: No locale for '${locales}', falling back to en`);
activateLocale(DEFAULT_FALLBACK());
}
export function activateLocale(code: string) {
const urlLocale = fromUrl("locale");
if (urlLocale !== null && urlLocale !== "") {
code = urlLocale;
}
const locale = LOCALES.find((locale) => locale.code == code);
if (!locale) {
console.warn(`authentik/locale: failed to find locale for code ${code}`);
return;
}
locale.locale().then((localeData) => {
console.debug(`authentik/locale: Loaded locale '${code}'`);
if (i18n.locale === code) {
return;
}
i18n.load(locale.code, localeData.locale);
i18n.activate(locale.code);
window.dispatchEvent(
new CustomEvent(EVENT_LOCALE_CHANGE, {
bubbles: true,
composed: true,
}),
);
});
}

26
web/src/common/ui/locale/activateLocale.ts Normal file
View File

@ -0,0 +1,26 @@
import { getLocale, setLocale } from "./configureLocale";
import { getBestMatchLocale, localeFromUrl } from "./helpers";
export function activateLocale(code: string) {
const urlLocale = localeFromUrl("locale");
if (urlLocale !== null && urlLocale !== "") {
code = urlLocale;
}
const locale = getBestMatchLocale(code);
if (!locale) {
console.warn(`authentik/locale: failed to find locale for code ${code}`);
return;
}
locale.locale().then(() => {
console.debug(`authentik/locale: Loaded locale '${code}'`);
if (getLocale() === code) {
return;
}
console.debug(`Setting Locale to ... ${locale.label()} (${locale.code})`);
setLocale(locale.code);
});
}
export default activateLocale;

38
web/src/common/ui/locale/autodetectLanguage.ts Normal file
View File

@ -0,0 +1,38 @@
import { globalAK } from "@goauthentik/common/global";
import { activateLocale } from "./activateLocale";
import { setLocale } from "./configureLocale";
import { DEFAULT_FALLBACK } from "./definitions";
import { findSupportedLocale, localeFromUrl } from "./helpers";
const isLocaleCandidate = (v: unknown): v is string => typeof v === "string" && v !== "";
export function autoDetectLanguage(defaultLanguage = "en") {
// Always load en locale at the start so we have something and don't error
setLocale(defaultLanguage);
// Get all locales we can, in order
// - Global authentik settings (contains user settings)
// - URL parameter
// - Navigator
// - Fallback (en)
const localeCandidates: string[] = [
globalAK()?.locale,
localeFromUrl("locale"),
window.navigator.language,
DEFAULT_FALLBACK,
].filter(isLocaleCandidate);
const firstSupportedLocale = findSupportedLocale(localeCandidates);
if (!firstSupportedLocale) {
console.debug(`authentik/locale: No locale for '${localeCandidates}', falling back to en`);
activateLocale(defaultLanguage);
return;
}
activateLocale(firstSupportedLocale.code);
}
export default autoDetectLanguage;

17
web/src/common/ui/locale/configureLocale.ts Normal file
View File

@ -0,0 +1,17 @@
import { configureLocalization } from "@lit/localize";
import { sourceLocale, targetLocales } from "../../../locale-codes";
import { getBestMatchLocale } from "./helpers";
export const { getLocale, setLocale } = configureLocalization({
sourceLocale,
targetLocales,
loadLocale: async (locale: string) => {
const localeDef = getBestMatchLocale(locale);
if (!localeDef) {
console.warn(`Unrecognized locale: ${localeDef}`);
return Promise.reject("");
}
return localeDef.locale();
},
});

61
web/src/common/ui/locale/definitions.ts Normal file
View File

@ -0,0 +1,61 @@
import * as _enLocale from "@goauthentik/locales/en";
import type { LocaleModule } from "@lit/localize";
import { msg } from "@lit/localize";
import { AkLocale, LocaleRow } from "./types";
export const DEFAULT_FALLBACK = "en";
const enLocale: LocaleModule = _enLocale;
export { enLocale };
// NOTE: This table cannot be made any shorter, despite all the repetition of syntax. Bundlers look
// for the `await import` string as a *string target* for doing alias substitution, so putting
// the import in some sort of abstracting function doesn't work. The same is true for the `msg()`
// function, which `localize` uses to find strings for extraction and translation. Likewise,
// because this is a file-level table, the `msg()` must be thunked so that they're re-run when
// the user changes the locale.
// NOTE: The matchers try to conform loosely to [RFC
// 5646](https://www.rfc-editor.org/rfc/rfc5646.txt), "Tags for the Identification of Languages." In
// practice, language tags have been seen using both hyphens and underscores, and the Chinese
// language uses both "regional" and "script" suffixes. The regexes use the language and any region
// or script.
//
// French is currently an oddity; the translator provided the France regional version explicitly,
// and we fall back to that regardless of region. Sorry, Québécois.
//
// Chinese locales usually (but not always) use the script rather than region suffix. The default
// (optional) fallback for Chinese (zh) is "Chinese (simplified)", which is why it has that odd
// regex syntax at the end which means "match zh as long as it's not followed by a [:word:] token";
// Traditional script and the Taiwanese are attempted first, and if neither matches, anything
// beginning with that generic "zh" is mapped to "Chinese (simplified)."
// - Code for Lit/Locale
// - Regex for matching user-supplied locale.
// - Text Label
// - Locale loader.
// prettier-ignore
const LOCALE_TABLE: LocaleRow[] = [
["en", /^en([_-]|$)/i, () => msg("English"), async () => await import("@goauthentik/locales/en")],
["es", /^es([_-]|$)/i, () => msg("Spanish"), async () => await import("@goauthentik/locales/es")],
["de", /^de([_-]|$)/i, () => msg("German"), async () => await import("@goauthentik/locales/de")],
["fr_FR", /^fr([_-]|$)/i, () => msg("French"), async () => await import("@goauthentik/locales/fr_FR")],
["pl", /^pl([_-]|$)/i, () => msg("Polish"), async () => await import("@goauthentik/locales/pl")],
["tr", /^tr([_-]|$)/i, () => msg("Turkish"), async () => await import("@goauthentik/locales/tr")],
["zh-Hant", /^zh[_-](HK|Hant)/i, () => msg("Chinese (traditional)"), async () => await import("@goauthentik/locales/zh-Hant")],
["zh_TW", /^zh[_-]TW$/i, () => msg("Taiwanese Mandarin"), async () => await import("@goauthentik/locales/zh_TW")],
["zh-Hans", /^zh(\b|_)/i, () => msg("Chinese (simplified)"), async () => await import("@goauthentik/locales/zh-Hans")],
];
export const LOCALES: AkLocale[] = LOCALE_TABLE.map(([code, match, label, locale]) => ({
code,
match,
label,
locale,
}));
export default LOCALES;

31
web/src/common/ui/locale/helpers.ts Normal file
View File

@ -0,0 +1,31 @@
import { LOCALES as RAW_LOCALES, enLocale } from "./definitions";
import { AkLocale } from "./types";
// NOTE: This is the definition of the LOCALES table that most of the code uses. The 'definitions'
// file is relatively pure, but here we establish that we want the English locale to loaded when an
// application is first instantiated.
export const LOCALES = RAW_LOCALES.map((locale) =>
locale.code === "en" ? { ...locale, locale: async () => enLocale } : locale,
);
// First attempt a precise match, then see if there's a precise match on the requested locale's
// prefix, then find the *first* locale for which that locale's prefix matches the requested prefix.
export function getBestMatchLocale(locale: string): AkLocale | undefined {
return LOCALES.find((l) => l.match.test(locale));
}
// This looks weird, but it's sensible: we have several candidates, and we want to find the first
// one that has a supported locale. Then, from *that*, we have to extract that first supported
// locale.
export function findSupportedLocale(candidates: string[]) {
const candidate = candidates.find((candidate: string) => getBestMatchLocale(candidate));
return candidate ? getBestMatchLocale(candidate) : undefined;
}
export function localeFromUrl(param = "locale") {
const url = new URL(window.location.href);
return url.searchParams.get(param) || "";
}

6
web/src/common/ui/locale/index.ts Normal file
View File

@ -0,0 +1,6 @@
import activateLocale from "./activateLocale";
import autoDetectLanguage from "./autodetectLanguage";
import { getLocale, setLocale } from "./configureLocale";
import { LOCALES } from "./helpers";
export { LOCALES, getLocale, setLocale, activateLocale, autoDetectLanguage };

10
web/src/common/ui/locale/types.ts Normal file
View File

@ -0,0 +1,10 @@
import type { LocaleModule } from "@lit/localize";
export type LocaleRow = [string, RegExp, () => string, () => Promise<LocaleModule>];
export type AkLocale = {
code: string;
match: RegExp;
label: () => string;
locale: () => Promise<LocaleModule>;
};

4
web/src/common/ws.ts
View File

@ -1,7 +1,7 @@
import { EVENT_MESSAGE, EVENT_WS_MESSAGE } from "@goauthentik/common/constants";
import { MessageLevel } from "@goauthentik/common/messages";
import { t } from "@lingui/macro";
import { msg } from "@lit/localize";
export interface WSMessage {
message_type: string;
@ -38,7 +38,7 @@ export class WebsocketClient {
composed: true,
detail: {
level: MessageLevel.error,
message: t`Connection error, reconnecting...`,
message: msg("Connection error, reconnecting..."),
},
}),
);