Addressing
Running clj -X:deps tree in bin/build-drivers the cli version is getting picked up
as I switched clojure-cli versions between 1.11 and 1.10.

I think this is because including metabase/metabase-core as a local/root no longer
places the clojure dep within it at the top level.

Now, the docs say
> It is a top dep (top dep versions always win) or it is a new lib or a newer version of a known lib (Maven keeps only the first found version regardless of new/old)

And so I would expect to see org.clojure/clojure in the deps tree under
metabase/metabase-core but it's missing, which to me means clojure maybe treated
specially if the root deps don't specify it.

This fixes many minor translation errors where apostrohpes are missing in the rendered message. This is extremely
common in French translations. In some cases it even causes {0} params to be missed since they accidentally end up
inside an improperly escaped string.

The i18n in Metabase uses a service 'POEditor', which requires:

1. all strings on the frontend and backend that need to have translations must be collected into a .pot file
2. The .pot file is uploaded and people contribute translations for each phrase, per locale. These eventually get
downloaded as locale.po (eg. German is 'de.po')
3. Frontend and backend each take responsibility for generating required locale resources, which are then used to
render the correct translations for the user's locale in the app.

This process works just fine, but occasionally some bugs are found. These particular changes in this PR were found
because of issue #15660 where a user reports that some of the 'binning options' are translated while others are not.

Those particular strings originate form the backend via the API at `src/metabase/api/table.clj`, but the issue arises
because the backend's locale resource files do not contain the entries for those translations. The mechanism is
working just fine, it's the resource building process that has some issues.

So, we can turn to the i18n build process, which is all found in `bin/i18n`, and is a mix of shell scripts and
clojure.

The script `update-translation-template` starts the process.
There are a few steps in the process:

1. generate metabase-frontend.pot -> uses Babel, I'm assuming this works for the purpose of this PR

2. generate metabase-backend.pot  -> works fine. We can see in `locales/metabase-backend.pot`:

   #: src/metabase/api/table.clj
   msgid "Week"
   msgstr ""

3. Join these files together with a tool 'msgcat' into metabase.pot
   - here is where we begin to see an issue.
   - there is NO entry that matches the example from 2. This is because msgcat has combined translation strings that
   match on the frontend into a single entry:

     #: frontend/src/metabase-lib/lib/Dimension.ts:1677
     #: frontend/src/metabase/lib/query_time.js:203
     #: frontend/src/metabase/parameters/components/widgets/DateRelativeWidget.jsx:25
     #: frontend/src/metabase/parameters/components/widgets/DateRelativeWidget.jsx:30
     #: src/metabase/api/table.clj
     msgid "Week"
     msgstr ""

   - and, more interestingly:

   #: frontend/src/metabase-lib/lib/Dimension.ts:1689
   #: frontend/src/metabase/lib/query_time.js:207 src/metabase/api/table.clj
   msgid "Quarter"
   msgstr ""

   - There are a bunch of #: lines, the last being the clj file, which means that "Week" is used both in the frontend
   and backend. So, the translated string can be used in both situations. The backend does handle the case of multiple
   #: paths, but does not handle the second situation, where there are actually 2 paths (space separated) in a single
   #: line.

4. in the file `bin/i18n/src/create_artifacts/backend.clj` we create .edn files for each locale.
   - this is a map where the keys are the English strings and values are the translations from the locale.po file.
   - our logic in this file incorrectly assumes only a single path per #: line in the .pot file exists, and this is
   one place where things break. The 'backend-message?' predicate will fail on the second example in 3. so we don't
   ever translate that string.
   - Simultaneously, `i18n/common.clj` allows pluralized entries. In the case of "Week", we got "Week" and "Weeks"
   translated from POEditor. When a message is plural, the contents show up in a different location. Our backend code
   ignores plural, but this ends up throwing away our valid translation as well. When a message is plural, all
   translations show up in :str-plural as a seq, where the first entry is the singular translation. But, we (remove
   :pluarl?) so get nothing. This is why the binning options aren't translated. As a point of completeness: if a
   message has no plural, we correctly get the translated message from the :str key.

The broken logic:
- common.clj - po-messages-seq creates a seq of messages, where a message looks like:
{:id "Week",
  :id-plural "Weeks",
  :str nil,
  :str-plural ("Woche" "Wochen"),
  :fuzzy? false,
  :plural? true,
  :source-references
  ("frontend/src/metabase/lib/query_time.js:203"
   "frontend/src/metabase/parameters/components/widgets/DateRelativeWidget.jsx:25"
   "frontend/src/metabase/parameters/components/widgets/DateRelativeWidget.jsx:30"
   "src/metabase/api/table.clj target/classes/metabase/api/table.clj"),
  :comment nil}

or
{:id "Day of Week",
  :id-plural nil,
  :str "Wochentag",
  :str-plural nil,
  :fuzzy? false,
  :plural? false,
  :source-references ("src/metabase/api/table.clj target/classes/metabase/api/table.clj"),
  :comment nil}

Example 1 fails because the message is pluralized. :str is nil, but (first (:str-plural message)) has the correct
translation. We are looking in the wrong spot in this case. Compare to Example 2 which does work.

Notice a subtle thing that was actually wrong in examples 1 and 2 but worked by accident: some :source-references
strings contain 2 paths. We see this failing here:

{:id "Quarter",
 :id-plural "Quarters",
 :str nil,
 :str-plural ("Quartal" "Quartale"),
 :fuzzy? false,
 :plural? true,
 :source-references
 ("frontend/src/metabase/lib/query_time.js:207 src/metabase/api/table.clj" "target/classes/metabase/api/table.clj"),
 :comment nil}

We can solve this different problem by using str/split in the backend-message? predicate.

* Make namespace aliasing consistent everywhere; enforce with clj-kondo

See the table of aliases in .clj-kondo/config.edn

Notable patterns:
- `[metabase.api.foo :as api.foo]`
- `[metabase.models.foo :as foo]`
- `[metabase.query-processor.foo :as qp.foo]`
- `[metabase.server.middleware.foo :as mw.foo]`
- `[metabase.util.foo :as u.foo]`
- `[clj-http.client :as http]` and `[metabase.http-client :as client]`

Fixes #19930.

Co-authored-by: Ryan Laurie <iethree@gmail.com>

Upgrade Liquibase to latest version; remove final Java source file and need for `clojure -X:deps prep` (#20611)

* Upgrade Liquibase to latest version

* Try adjusting log

* Fix checksums for the TWO migrations with ID = 32

* FINALLY get Liquibase to use Log4j2

* Set Liquibase ConsoleUIService OutputStream to null OutputStream

* Manually define a package for our H2 proxy class so Java 8 works

* Fix package-name determination code

* Update migrations file spec

* `databasechangelog` shouldn't be upper-case

* Lower-case quartz table names

* More MySQL fixes 

* Properties for all the Quartz tables 

* Formatting tweaks [ci skip]

* Revert a few more busted changes

* Fix more busted changes

* Bump Liquibase version to 4.8.0 to fix MySQL defaultValueBoolean bug

* OMG I think I finally fixed MySQL

* Remove Java source file and prep-deps code

* Remove two more references to bin/prep.sh

* Minor cleanup

* Revert unneeded changes

* Fix busted indentation

* Don't search inside java/ anymore since it's G-O-N-E

* Appease the namespace linter

* Update src/metabase/db/liquibase/h2.clj

* Bump SAML lib version

* Add entry to recognize metabase/saml20-clj as being released under EPL v2

* Replace AOT Spark SQL deps with a `proxy` and a `DataSource`

* Support `connection-details->spec` returning a `DataSource`

* Remove target/classes

* Don't need to deps prep drivers anymore

* Fix duplicate `this` params in `proxy` methods; add `:test` alias for Eastwood to make it be a little quieter

* Make sure to call `pool/map->properties`

* Upgrade Hive JDBC driver version from 1.2.2 -> 3.1.2; Bump Spark SQL from 2.1.1 to 3.2.1

* Clean the namespaces

* Don't need to register the a proxy JDBC driver since we're not even using it anymore

* Fix Spark SQL :schema sync for newer versions

* Remove unneeded override

* Fix day-of-week extract for new :sparksql

* Hive/Spark SQL needs to escape question marks inside QUOTED identifiers now 

* Some minor SQL generation improvements to avoid duplicate casts

* Revert change to debug test

* Fix locking in new Alpine release container
Seems like the new Alpine container (3.15) has some issue when doing apk upgrade first which ends up erroring out the process, so moving it to the last part of the process (after installing the packages) makes the trick.
You can test it out by using the current process (errors with a "Unable to lock database: temporary error (try again later)") and the one in this branch which will finish successfully

* Bump backend dependencies (Jan 2022)

* Revert java-time version upgrade for now until https://github.com/dm3/clojure.java-time/issues/77 is fixed

* Add license overrides

* Bump a few more deps (again). Revert Google/BigQuery and Vertica version bumps

* Revert MariaDB and Redshift version changes

0.42 translations: adds Indonesian, cleans up some strings, and updates our i18n build script (#20041)

* update translation build script to catch ts and tsx files

* add blank file for indonesian

* makes password resetting less awesome

* get rid of 'this item' text

* add updated translations

* Adjust E2E tests expectation of the strings (after the modification)

Co-authored-by: Ariya Hidayat <ariya@metabase.com>

* Support overriding ROWCOUNT for SQL Server

Add new "ROWCOUNT Override" connection property for `:sqlserver`, which will provide a DB-level mechanism to override the `ROWCOUNT` session level setting as needed for specific DBs

Change `max-results-bare-rows` from a hardcoded constant to a setting definition instead, which permits a DB level override, and move the former constant default to a new def instead (`default-max-results-bare-rows`)

For `:sqlserver`, set the DB-level setting override (if the connection property is set), via the `driver/normalize-db-details` impl

Add test to confirm the original scenario from #9940 works using this new override (set to `0`)

Move common computation function of overall row limit to the `metabase.query-processor.middleware.limit` namespace, and invoke it from execute now, called `determine-query-max-rows`

Add new clause to the `determine-query-max-rows` function that preferentially takes the value from `row-limit-override` (if defined)

* Switch to Temurin official image

This image is a pure musl alpine image

We will fall back to the manual workaround of tweaking the commit
timestamp.

Add functions to describe-database namespace for performing inclusive/exclusive schema filtering

Add tests for these functions in isolation

Update `:bigquery-cloud-sdk` driver and QP code to remove the single `dataset-id` conn-param

Changing :bigquery-cloud-sdk` driver to use dataset inclusion/exclusion filters instead

Updating `add.cy.spec.js` Cypress test to account for new structure

Add data upgrade (via `driver/normalize-db-details` implementation) to change `dataset-id` property to inclusion filter

Add test to confirm data upgrade

Adding server side expansion of `:schema-filters` type connection property

We had some CI breaks that are predicated on the year changing. Fix them.

* Add checks for _FILE env vars

* Fixing this for once and for all + documenting

* Update run_metabase.sh

Updating release script with a new build option to force the release to be latest (by overriding the GIT_COMMITTER_DATE env var when creating the tag)

Don't use yarn here since it will fetch dependencies specified in
package.json. Instead, use npm so that dependencies frozen in
package-lock.json, which metabase.github.io uses, are downloaded instead.

Add new deps for finding close misspellings of map keys (spell-spec) and printing more readable spec errors (expound)

Adding spec definitions for driver YAML format

Add call to driver YAML validation from verify.clj

Fixing a couple issues in existing drivers found by the validation

* Add index on QueryExecution card_id + started_at

* Migrations linter shouldn't disallow legacymigrations > 382

* Remove old test

* Change all active TEXT columns in MySQL app DB to LONGTEXT

Add new text.type Liquibase property to dynamically select the correct text type to use, per DB (which is LONGTEXT for MySQL), very similar to the existing blob.type one

Adding new migrations that update all existing app DB TEXT columns to LONGTEXT columns, only for mariadb/mysql

Update migrations linter to ensure no new text types get added, via a new predicate that searches for text types being added

Do the same logic for "blob" now (should be "${blob.type}"), update rule and test

Add test for migrations, that checks the new types, for all types in scope for this PR, to ensure that they have all been changed to their expected DB-specific type (either LONGTEXT for MySQL or TEXT/CLOB for others)

Updating a couple migrations to remove special casing that was originally done only for MySQL to simply make them universally "${text.type}", in order to unify behavior and reduce surprises going forward

Co-authored-by: Cam Saul <github@camsaul.com>

* Revert changes from Jeff's PR

* Add index to ModerationReview moderated_item_type + moderated_item_id

* Add to 0.41.2 instead

* Require explicit index name for createIndex

* Move 41.2 migrations to after the 41.0 migrations

* Adopt new migration numbering scheme

* Fix comments

* Fix MySQL + MariaDB insanity

* Fix ID range validation

* Actually 382 is the last legacy ID

* Improved validation and tests

* Adopt the new-new migration ID format.

* Test fixes 

* Adopt new migration numbering scheme

* Fix comments

* Fix MySQL + MariaDB insanity

* Fix ID range validation

* Actually 382 is the last legacy ID

* Improved validation and tests

* Adopt the new-new migration ID format.

* Test fixes 

* Fix precondition of change set 97

Without the `type` and with the space Liquibase is unable to parse this
precondition.

During `lein test` it outputs:
```
[clojure-agent-send-off-pool-0] DEBUG liquibase.changelog - Running Changeset:migrations/000_migrations.yaml::97::senior
[clojure-agent-send-off-pool-0] DEBUG liquibase.executor - Changeset migrations/000_migrations.yaml::97::senior
[clojure-agent-send-off-pool-0] DEBUG liquibase.executor - Added 0.32.0
[clojure-agent-send-off-pool-0] INFO  liquibase.changelog - Marking ChangeSet: migrations/000_migrations.yaml::97::senior ran despite precondition failure due to onFail='MARK_RAN':
          liquibase.yaml : DBMS Precondition failed: expected null, got h2

[clojure-agent-send-off-pool-0] DEBUG liquibase.changelog - Skipping ChangeSet: migrations/000_migrations.yaml::97::senior
[clojure-agent-send-off-pool-0] DEBUG liquibase.executor - Executing with the 'jdbc' executor
[clojure-agent-send-off-pool-0] DEBUG liquibase.executor - 1 row(s) affected
```

After this change the output changes to:
```
[clojure-agent-send-off-pool-0] DEBUG liquibase.changelog - Running Changeset:migrations/000_migrations.yaml::97::senior
[clojure-agent-send-off-pool-0] DEBUG liquibase.executor - Changeset migrations/000_migrations.yaml::97::senior
[clojure-agent-send-off-pool-0] DEBUG liquibase.executor - Added 0.32.0
[clojure-agent-send-off-pool-0] INFO  liquibase.changelog - Marking ChangeSet: migrations/000_migrations.yaml::97::senior ran despite precondition failure due to onFail='MARK_RAN':
          liquibase.yaml : DBMS Precondition failed: expected mysql,mariadb, got h2

[clojure-agent-send-off-pool-0] DEBUG liquibase.changelog - Skipping ChangeSet: migrations/000_migrations.yaml::97::senior
[clojure-agent-send-off-pool-0] DEBUG liquibase.executor - Executing with the 'jdbc' executor
[clojure-agent-send-off-pool-0] DEBUG liquibase.executor - 1 row(s) affected
```

For documentation of the syntax cf.
 https://docs.liquibase.com/concepts/advanced/preconditions.html



* Extend migration linter to check dbms preconditions

* Also validate the `type` field of the `dbms` precondition

Co-authored-by: dpsutton <dan@dpsutton.com>

NODE_ENV interferes with other tools (e.g. `yarn` won't pull
devDependencies). Since what we need is only a flag indicating Webpack
to bundle for production vs developement, invent our own env.

This also solves the mystery of unable to call `yarn build` in the build
script `bin/build-mb/src/build.clj`.

This is a combination of 2 issues:
 1) pt got renamed to pt_BR in x.39.x but the old
    "pt" value may still be stored in the database
 2) when making a release, an unclean build
    directory may be used containing old "pt"
    resource which gets leaked into the release build

1) is fixed by adding a function to treat "pt" as "pt_BR"
to support users who were on "pt" since pre-x.39.
(This is done by finding the closest fallback locale)

2) is fixed by emptying the folder before generating locales
so any old locales are deleted.

Fixes #16690

The ttag library expect the plural array to be exactly the size as
the number of plural forms defined in the header.

If we remove empty plural, the array will have the wrong size and
the library will crash when the trying to use the non-existent
plural.

By leaving the empty string there, the ttag library will correctly
detect it and use the English version for the missing variant.

This does not affect the backend because the backend does not
support plurals.

Fixes #16323

Make sure built drivers do not contain clojure core classes. Handle either string or keyword args to build scripts (#17608)

* No Clojure core classes please

* Don't try to make paths absolute twice. Build scripts should handle keyword args like ':driver'

* parse-as-keyword should return keywords as is

* Test fix 

* Test fix 

* Test fix part 2

* Test init code: read driver parents from driver deps.edn

* Rename :metabase.build-driver/parents -> :metabase.driver/parents

* Determine MB project root directory relative to source directory of build scripts

* Support calling build-driver! as a -X fn, and support dirs outside of modules/drivers

* Support loading 3rd-party driver manifest files at (dev) launch

* Fix log4j not working on latest version of CIDER (unrelated)

* Fix dissoc

* project-root-directory is supposed to be a string

New BigQuery Driver

Create new :bigquery-cloud-sdk driver using the google-cloud-bigquery library instead, and whose source is adapted from the :bigquery driver

https://cloud.google.com/bigquery/docs/reference/libraries

Marking existing :bigquery driver as deprecated, and superseded-by the new one (bigquery-cloud-sdk)

Update new driver and query processor code to use newer Google SDK

Switch test data loading over to use new API

Add project-id connection property to override the value from the service account JSON, and use it as part of qualified names in the query processor if set

Updating google driver so its libraries are compatible with the newer ones used in BigQuery

Update date bucketing tests to skip :bigquery-cloud-sdk (new driver) where :bigquery is skipped

Update `with-bigquery-fks` to take in the driver, since there are now multiple ones

Adding test to confirm that overriding project-id for a public BQ project works (sync and query)

Fixing a bunch of whitespace alignment errors in tests