This fixes many minor translation errors where apostrohpes are missing in the rendered message. This is extremely
common in French translations. In some cases it even causes {0} params to be missed since they accidentally end up
inside an improperly escaped string.

The i18n in Metabase uses a service 'POEditor', which requires:

1. all strings on the frontend and backend that need to have translations must be collected into a .pot file
2. The .pot file is uploaded and people contribute translations for each phrase, per locale. These eventually get
downloaded as locale.po (eg. German is 'de.po')
3. Frontend and backend each take responsibility for generating required locale resources, which are then used to
render the correct translations for the user's locale in the app.

This process works just fine, but occasionally some bugs are found. These particular changes in this PR were found
because of issue #15660 where a user reports that some of the 'binning options' are translated while others are not.

Those particular strings originate form the backend via the API at `src/metabase/api/table.clj`, but the issue arises
because the backend's locale resource files do not contain the entries for those translations. The mechanism is
working just fine, it's the resource building process that has some issues.

So, we can turn to the i18n build process, which is all found in `bin/i18n`, and is a mix of shell scripts and
clojure.

The script `update-translation-template` starts the process.
There are a few steps in the process:

1. generate metabase-frontend.pot -> uses Babel, I'm assuming this works for the purpose of this PR

2. generate metabase-backend.pot  -> works fine. We can see in `locales/metabase-backend.pot`:

   #: src/metabase/api/table.clj
   msgid "Week"
   msgstr ""

3. Join these files together with a tool 'msgcat' into metabase.pot
   - here is where we begin to see an issue.
   - there is NO entry that matches the example from 2. This is because msgcat has combined translation strings that
   match on the frontend into a single entry:

     #: frontend/src/metabase-lib/lib/Dimension.ts:1677
     #: frontend/src/metabase/lib/query_time.js:203
     #: frontend/src/metabase/parameters/components/widgets/DateRelativeWidget.jsx:25
     #: frontend/src/metabase/parameters/components/widgets/DateRelativeWidget.jsx:30
     #: src/metabase/api/table.clj
     msgid "Week"
     msgstr ""

   - and, more interestingly:

   #: frontend/src/metabase-lib/lib/Dimension.ts:1689
   #: frontend/src/metabase/lib/query_time.js:207 src/metabase/api/table.clj
   msgid "Quarter"
   msgstr ""

   - There are a bunch of #: lines, the last being the clj file, which means that "Week" is used both in the frontend
   and backend. So, the translated string can be used in both situations. The backend does handle the case of multiple
   #: paths, but does not handle the second situation, where there are actually 2 paths (space separated) in a single
   #: line.

4. in the file `bin/i18n/src/create_artifacts/backend.clj` we create .edn files for each locale.
   - this is a map where the keys are the English strings and values are the translations from the locale.po file.
   - our logic in this file incorrectly assumes only a single path per #: line in the .pot file exists, and this is
   one place where things break. The 'backend-message?' predicate will fail on the second example in 3. so we don't
   ever translate that string.
   - Simultaneously, `i18n/common.clj` allows pluralized entries. In the case of "Week", we got "Week" and "Weeks"
   translated from POEditor. When a message is plural, the contents show up in a different location. Our backend code
   ignores plural, but this ends up throwing away our valid translation as well. When a message is plural, all
   translations show up in :str-plural as a seq, where the first entry is the singular translation. But, we (remove
   :pluarl?) so get nothing. This is why the binning options aren't translated. As a point of completeness: if a
   message has no plural, we correctly get the translated message from the :str key.

The broken logic:
- common.clj - po-messages-seq creates a seq of messages, where a message looks like:
{:id "Week",
  :id-plural "Weeks",
  :str nil,
  :str-plural ("Woche" "Wochen"),
  :fuzzy? false,
  :plural? true,
  :source-references
  ("frontend/src/metabase/lib/query_time.js:203"
   "frontend/src/metabase/parameters/components/widgets/DateRelativeWidget.jsx:25"
   "frontend/src/metabase/parameters/components/widgets/DateRelativeWidget.jsx:30"
   "src/metabase/api/table.clj target/classes/metabase/api/table.clj"),
  :comment nil}

or
{:id "Day of Week",
  :id-plural nil,
  :str "Wochentag",
  :str-plural nil,
  :fuzzy? false,
  :plural? false,
  :source-references ("src/metabase/api/table.clj target/classes/metabase/api/table.clj"),
  :comment nil}

Example 1 fails because the message is pluralized. :str is nil, but (first (:str-plural message)) has the correct
translation. We are looking in the wrong spot in this case. Compare to Example 2 which does work.

Notice a subtle thing that was actually wrong in examples 1 and 2 but worked by accident: some :source-references
strings contain 2 paths. We see this failing here:

{:id "Quarter",
 :id-plural "Quarters",
 :str nil,
 :str-plural ("Quartal" "Quartale"),
 :fuzzy? false,
 :plural? true,
 :source-references
 ("frontend/src/metabase/lib/query_time.js:207 src/metabase/api/table.clj" "target/classes/metabase/api/table.clj"),
 :comment nil}

We can solve this different problem by using str/split in the backend-message? predicate.

* Make namespace aliasing consistent everywhere; enforce with clj-kondo

See the table of aliases in .clj-kondo/config.edn

Notable patterns:
- `[metabase.api.foo :as api.foo]`
- `[metabase.models.foo :as foo]`
- `[metabase.query-processor.foo :as qp.foo]`
- `[metabase.server.middleware.foo :as mw.foo]`
- `[metabase.util.foo :as u.foo]`
- `[clj-http.client :as http]` and `[metabase.http-client :as client]`

Fixes #19930.

Co-authored-by: Ryan Laurie <iethree@gmail.com>

Upgrade Liquibase to latest version; remove final Java source file and need for `clojure -X:deps prep` (#20611)

* Upgrade Liquibase to latest version

* Try adjusting log

* Fix checksums for the TWO migrations with ID = 32

* FINALLY get Liquibase to use Log4j2

* Set Liquibase ConsoleUIService OutputStream to null OutputStream

* Manually define a package for our H2 proxy class so Java 8 works

* Fix package-name determination code

* Update migrations file spec

* `databasechangelog` shouldn't be upper-case

* Lower-case quartz table names

* More MySQL fixes 

* Properties for all the Quartz tables 

* Formatting tweaks [ci skip]

* Revert a few more busted changes

* Fix more busted changes

* Bump Liquibase version to 4.8.0 to fix MySQL defaultValueBoolean bug

* OMG I think I finally fixed MySQL

* Remove Java source file and prep-deps code

* Remove two more references to bin/prep.sh

* Minor cleanup

* Revert unneeded changes

* Fix busted indentation

* Don't search inside java/ anymore since it's G-O-N-E

* Appease the namespace linter

* Update src/metabase/db/liquibase/h2.clj

* Bump SAML lib version

* Add entry to recognize metabase/saml20-clj as being released under EPL v2

* Replace AOT Spark SQL deps with a `proxy` and a `DataSource`

* Support `connection-details->spec` returning a `DataSource`

* Remove target/classes

* Don't need to deps prep drivers anymore

* Fix duplicate `this` params in `proxy` methods; add `:test` alias for Eastwood to make it be a little quieter

* Make sure to call `pool/map->properties`

* Upgrade Hive JDBC driver version from 1.2.2 -> 3.1.2; Bump Spark SQL from 2.1.1 to 3.2.1

* Clean the namespaces

* Don't need to register the a proxy JDBC driver since we're not even using it anymore

* Fix Spark SQL :schema sync for newer versions

* Remove unneeded override

* Fix day-of-week extract for new :sparksql

* Hive/Spark SQL needs to escape question marks inside QUOTED identifiers now 

* Some minor SQL generation improvements to avoid duplicate casts

* Revert change to debug test

* Fix locking in new Alpine release container
Seems like the new Alpine container (3.15) has some issue when doing apk upgrade first which ends up erroring out the process, so moving it to the last part of the process (after installing the packages) makes the trick.
You can test it out by using the current process (errors with a "Unable to lock database: temporary error (try again later)") and the one in this branch which will finish successfully

* Bump backend dependencies (Jan 2022)

* Revert java-time version upgrade for now until https://github.com/dm3/clojure.java-time/issues/77 is fixed

* Add license overrides

* Bump a few more deps (again). Revert Google/BigQuery and Vertica version bumps

* Revert MariaDB and Redshift version changes

0.42 translations: adds Indonesian, cleans up some strings, and updates our i18n build script (#20041)

* update translation build script to catch ts and tsx files

* add blank file for indonesian

* makes password resetting less awesome

* get rid of 'this item' text

* add updated translations

* Adjust E2E tests expectation of the strings (after the modification)

Co-authored-by: Ariya Hidayat <ariya@metabase.com>

* Support overriding ROWCOUNT for SQL Server

Add new "ROWCOUNT Override" connection property for `:sqlserver`, which will provide a DB-level mechanism to override the `ROWCOUNT` session level setting as needed for specific DBs

Change `max-results-bare-rows` from a hardcoded constant to a setting definition instead, which permits a DB level override, and move the former constant default to a new def instead (`default-max-results-bare-rows`)

For `:sqlserver`, set the DB-level setting override (if the connection property is set), via the `driver/normalize-db-details` impl

Add test to confirm the original scenario from #9940 works using this new override (set to `0`)

Move common computation function of overall row limit to the `metabase.query-processor.middleware.limit` namespace, and invoke it from execute now, called `determine-query-max-rows`

Add new clause to the `determine-query-max-rows` function that preferentially takes the value from `row-limit-override` (if defined)

* Switch to Temurin official image

This image is a pure musl alpine image

We will fall back to the manual workaround of tweaking the commit
timestamp.

Add functions to describe-database namespace for performing inclusive/exclusive schema filtering

Add tests for these functions in isolation

Update `:bigquery-cloud-sdk` driver and QP code to remove the single `dataset-id` conn-param

Changing :bigquery-cloud-sdk` driver to use dataset inclusion/exclusion filters instead

Updating `add.cy.spec.js` Cypress test to account for new structure

Add data upgrade (via `driver/normalize-db-details` implementation) to change `dataset-id` property to inclusion filter

Add test to confirm data upgrade

Adding server side expansion of `:schema-filters` type connection property

We had some CI breaks that are predicated on the year changing. Fix them.

* Add checks for _FILE env vars

* Fixing this for once and for all + documenting

* Update run_metabase.sh

Updating release script with a new build option to force the release to be latest (by overriding the GIT_COMMITTER_DATE env var when creating the tag)

Don't use yarn here since it will fetch dependencies specified in
package.json. Instead, use npm so that dependencies frozen in
package-lock.json, which metabase.github.io uses, are downloaded instead.

Add new deps for finding close misspellings of map keys (spell-spec) and printing more readable spec errors (expound)

Adding spec definitions for driver YAML format

Add call to driver YAML validation from verify.clj

Fixing a couple issues in existing drivers found by the validation

* Add index on QueryExecution card_id + started_at

* Migrations linter shouldn't disallow legacymigrations > 382

* Remove old test

* Change all active TEXT columns in MySQL app DB to LONGTEXT

Add new text.type Liquibase property to dynamically select the correct text type to use, per DB (which is LONGTEXT for MySQL), very similar to the existing blob.type one

Adding new migrations that update all existing app DB TEXT columns to LONGTEXT columns, only for mariadb/mysql

Update migrations linter to ensure no new text types get added, via a new predicate that searches for text types being added

Do the same logic for "blob" now (should be "${blob.type}"), update rule and test

Add test for migrations, that checks the new types, for all types in scope for this PR, to ensure that they have all been changed to their expected DB-specific type (either LONGTEXT for MySQL or TEXT/CLOB for others)

Updating a couple migrations to remove special casing that was originally done only for MySQL to simply make them universally "${text.type}", in order to unify behavior and reduce surprises going forward

Co-authored-by: Cam Saul <github@camsaul.com>

* Revert changes from Jeff's PR

* Add index to ModerationReview moderated_item_type + moderated_item_id

* Add to 0.41.2 instead

* Require explicit index name for createIndex

* Move 41.2 migrations to after the 41.0 migrations

* Adopt new migration numbering scheme

* Fix comments

* Fix MySQL + MariaDB insanity

* Fix ID range validation

* Actually 382 is the last legacy ID

* Improved validation and tests

* Adopt the new-new migration ID format.

* Test fixes 

* Adopt new migration numbering scheme

* Fix comments

* Fix MySQL + MariaDB insanity

* Fix ID range validation

* Actually 382 is the last legacy ID

* Improved validation and tests

* Adopt the new-new migration ID format.

* Test fixes 

* Fix precondition of change set 97

Without the `type` and with the space Liquibase is unable to parse this
precondition.

During `lein test` it outputs:
```
[clojure-agent-send-off-pool-0] DEBUG liquibase.changelog - Running Changeset:migrations/000_migrations.yaml::97::senior
[clojure-agent-send-off-pool-0] DEBUG liquibase.executor - Changeset migrations/000_migrations.yaml::97::senior
[clojure-agent-send-off-pool-0] DEBUG liquibase.executor - Added 0.32.0
[clojure-agent-send-off-pool-0] INFO  liquibase.changelog - Marking ChangeSet: migrations/000_migrations.yaml::97::senior ran despite precondition failure due to onFail='MARK_RAN':
          liquibase.yaml : DBMS Precondition failed: expected null, got h2

[clojure-agent-send-off-pool-0] DEBUG liquibase.changelog - Skipping ChangeSet: migrations/000_migrations.yaml::97::senior
[clojure-agent-send-off-pool-0] DEBUG liquibase.executor - Executing with the 'jdbc' executor
[clojure-agent-send-off-pool-0] DEBUG liquibase.executor - 1 row(s) affected
```

After this change the output changes to:
```
[clojure-agent-send-off-pool-0] DEBUG liquibase.changelog - Running Changeset:migrations/000_migrations.yaml::97::senior
[clojure-agent-send-off-pool-0] DEBUG liquibase.executor - Changeset migrations/000_migrations.yaml::97::senior
[clojure-agent-send-off-pool-0] DEBUG liquibase.executor - Added 0.32.0
[clojure-agent-send-off-pool-0] INFO  liquibase.changelog - Marking ChangeSet: migrations/000_migrations.yaml::97::senior ran despite precondition failure due to onFail='MARK_RAN':
          liquibase.yaml : DBMS Precondition failed: expected mysql,mariadb, got h2

[clojure-agent-send-off-pool-0] DEBUG liquibase.changelog - Skipping ChangeSet: migrations/000_migrations.yaml::97::senior
[clojure-agent-send-off-pool-0] DEBUG liquibase.executor - Executing with the 'jdbc' executor
[clojure-agent-send-off-pool-0] DEBUG liquibase.executor - 1 row(s) affected
```

For documentation of the syntax cf.
 https://docs.liquibase.com/concepts/advanced/preconditions.html



* Extend migration linter to check dbms preconditions

* Also validate the `type` field of the `dbms` precondition

Co-authored-by: dpsutton <dan@dpsutton.com>

NODE_ENV interferes with other tools (e.g. `yarn` won't pull
devDependencies). Since what we need is only a flag indicating Webpack
to bundle for production vs developement, invent our own env.

This also solves the mystery of unable to call `yarn build` in the build
script `bin/build-mb/src/build.clj`.

This is a combination of 2 issues:
 1) pt got renamed to pt_BR in x.39.x but the old
    "pt" value may still be stored in the database
 2) when making a release, an unclean build
    directory may be used containing old "pt"
    resource which gets leaked into the release build

1) is fixed by adding a function to treat "pt" as "pt_BR"
to support users who were on "pt" since pre-x.39.
(This is done by finding the closest fallback locale)

2) is fixed by emptying the folder before generating locales
so any old locales are deleted.

Fixes #16690

The ttag library expect the plural array to be exactly the size as
the number of plural forms defined in the header.

If we remove empty plural, the array will have the wrong size and
the library will crash when the trying to use the non-existent
plural.

By leaving the empty string there, the ttag library will correctly
detect it and use the English version for the missing variant.

This does not affect the backend because the backend does not
support plurals.

Fixes #16323

Make sure built drivers do not contain clojure core classes. Handle either string or keyword args to build scripts (#17608)

* No Clojure core classes please

* Don't try to make paths absolute twice. Build scripts should handle keyword args like ':driver'

* parse-as-keyword should return keywords as is

* Test fix 

* Test fix 

* Test fix part 2

* Test init code: read driver parents from driver deps.edn

* Rename :metabase.build-driver/parents -> :metabase.driver/parents

* Determine MB project root directory relative to source directory of build scripts

* Support calling build-driver! as a -X fn, and support dirs outside of modules/drivers

* Support loading 3rd-party driver manifest files at (dev) launch

* Fix log4j not working on latest version of CIDER (unrelated)

* Fix dissoc

* project-root-directory is supposed to be a string

New BigQuery Driver

Create new :bigquery-cloud-sdk driver using the google-cloud-bigquery library instead, and whose source is adapted from the :bigquery driver

https://cloud.google.com/bigquery/docs/reference/libraries

Marking existing :bigquery driver as deprecated, and superseded-by the new one (bigquery-cloud-sdk)

Update new driver and query processor code to use newer Google SDK

Switch test data loading over to use new API

Add project-id connection property to override the value from the service account JSON, and use it as part of qualified names in the query processor if set

Updating google driver so its libraries are compatible with the newer ones used in BigQuery

Update date bucketing tests to skip :bigquery-cloud-sdk (new driver) where :bigquery is skipped

Update `with-bigquery-fks` to take in the driver, since there are now multiple ones

Adding test to confirm that overriding project-id for a public BQ project works (sync and query)

Fixing a bunch of whitespace alignment errors in tests

* Backend SVG rendering proof of concept [ci skip]

* Update cssbox to 5.0.0

* Render bar, line, and pie charts in js to svg

sparkline is now done in js, bar is now recognized and done in js, new
:categorical/donut as well

* Remove api route for render

* pass along render-type, not hardcoded to :inline

* Move bar chart above sparkline and remove line check

In order to introduce the bar chart type need it above the sparkline
check since it is otherwise the same except for display property of
the card. But lots of tests assume that this will get hit with a nil
display type set in testing so remove checking for `:line` allows all
the testing cases to hit the right type

* Fix tests now that bar graphs aren't html but images

* Include attachments for bar charts

* Move over to in-tree bundle

* Force everything [ci noskip]

trying to ensure that the built jar includes the newer
"resources/frontend_client/app/dist/lib-static-viz.bundle.js"

* Run `yarn build-static-viz` in backend-deps in CI

this js file is now a hard dependency of the backend so it fits in
this tsk. All such things that depend on the backend sources will need
it. Makes me think perhaps we want a checked in version but i'm not
sure yet.

* Look on classpath not filesystem for js bundle [ci noskip]

* Move yarn build-static-viz into the checkout step

* License information for antlr4-runtime

* create attachment for categorical donuts

* add ordinal legend to donuts (#17177)

* set widths of html image and svg image to 1200

* Revert "add ordinal legend to donuts (#17177)"

This reverts commit 1eb81d2e.

* Helper functions to render html easily

* readme in dev

* readme ensure that static viz bundle exists

* Cleanup ns after removing proxy

* Donut chart colors and legend (#17251)

* use external color map for fill per dimension

* Add support new color legend for donut

* Ensure text doesn't appear as link

entire thing is actually the body of a link tag for emails but we want
a decent text color rather than a default link color

* use chart colors from https://stats.metabase.com/_internal/colors



* Make checkers happy

- remove unused imports
- add a docstring
- don't shadow fn with a local

* cleanup ns import

* Remove reflective call

* Cleanup ns on correct branch

Co-authored-by: dan sutton <dan@dpsutton.com>

* X-axis: just use (approx) 5 ticks to avoid overlapping labels (#17287)

* increase gap between arcs (#17271)

* Set rendering hints on html->image

* ignore width for now and make them larger

* Ns deprecation and some cleanup

* make namespace checker happy

* Simple tests for detecting chart type

* Rename from poc

* Tests for scalar/smartscalar

* cleanup js svg namespace a bit

* Tests of svg engine

* ns sorting after renaming

* Unify our two different js engine usages

settled on the js context. Has typed returns `(.asString ^Value ...)`
instead of perhaps capturing std out?
https://www.graalvm.org/sdk/javadoc/org/graalvm/polyglot/Value.html



Context is a bit more friendly for getting source into it. One
downside is that the invocable bit isn't quite as nice. The old way
would return a java.util.functionFunction but the difference is

(.apply function (object-array args))

vs

(.execute fn-ref (object-array args))

* Don't io/resource the io/resource

* js engine tests

* Ns cleanup in js-svg

type hints in the js-engine ns mean we don't need as many classes from
polyglot here

* Cleanup of text, ns docstrings, alignment

* Fix fill->fill-opacity with parsed doc, not regex

* Make a single helper that loads a static viz bundle context

* Docstrings and make private in js-svg

* Sort imported classes in js-svg

* Make width passed down through rendering aparatus

- svgs are always rendered at 1200 for quality
- slack images of html are rendered at 1200 so that they can be zoomed
  in in the ui but slack automatically scales down
- email sends html and the full svg but includes img width tags so
  that is handled appropriately

* docstring

Co-authored-by: dan sutton <dan@dpsutton.com>
Co-authored-by: Kyle Doherty <5248953+kdoh@users.noreply.github.com>
Co-authored-by: Ariya Hidayat <ariya@metabase.com>

* Add the whitespace linter

* Fix whitespace linter errors [except for one file]

* Add a line that will intentionally break stuff to verify the linter is working.

* Ok, remove the line that caused the linter to fail.

* Use latest version of the linter

* Fix missing newline