Snippets Groups Projects

MLv2 — Migrate notebook's join step to MLv2 (#33078)

Anton Kulyk authored 1 year ago

* MLv2 Joins 1 — Basic features (#32903)

* Fix `FilterClause` return type

* Export `Joinable` type

* Add `Join` type to `replaceClause`

* Add `CardDisplayInfo` type

* Add `FilterOperatorDisplayInfo` type

* Add `JoinExternalOp` type

* Export `QueryColumnPickerProps`

* Implement basic MLv2 `JoinStep`

* Remove `null` condition hack

* Sync join strategy on external query update

* Clean `JoinStep.styled.tsx`

* Add `JoinConditionClause` type

* Fix join external op type-casting

* Use HTML button in `JoinTablePicker`

* Simplify `JoinStep` logic

* Add `Join` type to `removeClause`

* Extract `useJoin` hook

* Extract `useJoinCondition` hook

* Group `JoinStep` components into directories

* Fix new condition rendering hack

* Use `cursor: pointer` for join column picker

* Fix condition duplication

* Fix missing LHS/RHS column method arguments

* Simplify join condition assertions

* Fetch metadata for selected table

* Fix `joinConditionLHSColumns` usage

* Highlight selected condition columns

* Preselect right table database, filter databases

* Fix table filter

* Open LHS column picker when table is selected

* Use `joinLHSDisplayName`

* Patch test

* Fix text

* Open RHS column picker once the LHS one is selected

* Preselect models picker for model based questions

* Fix test

* Remove TODOs

* Fix popover ref

* Disallow removing condition columns

* Use "Previous results" if table name is unavailable

* Fix previous results shown too early

* Update `joinLHSDisplayName` usage

* MLv2 Joins 2 — Suggested condition (#32904)

* Use suggested join condition if available

* Remove `it.todo`

* MLv2 Joins 3 — Fields (#32905)

* Update `FieldsPicker` API

* Export `JoinFields` type

* Implement join fields selection

* Fix empty column list when need to fetch metadata

* Add missing tests

* Remove `it.todo`

* Extend tests

* Fix tests

* Fix race-condition when loading table metadata

* Make field picker's `isColumnSelected` required

* MLv2 Joins 4 — Multiple conditions (#32906)

* Implement multiple conditions

* Fix tests

* Remove irrelevant test

* Extract `JoinConditionRightPart`

* Fix temporal units sync for join columns

* MLv2 Joins 5 — Update condition UI (#32907)

* Add `removeCondition` to `useJoin`

* Sync `useJoinCondition` state

* Update join condition UI

* Improve a11y

* Tweak UI

* Fix color literal usage

* Update UI

* Fix read only mode (#32908)

* MLv2 Joins 7 — Fix notebook editor lifecycle issue (#32911)

* Allow keeping a step open on calling `updateQuery`

* Don't auto-close join notebook steps

* Make join steps open by default

* Remove join on table change

* Move open steps calc to `getInitialOpenSteps`

* Remove no longer relevant check

* Add a link to proper fix description

* Use MLv2 in `notebook/lib/steps`

* Update join's `isValid` check

* Remove redundant tests

* Remove more redundant tests

* Revert "Use MLv2 in `notebook/lib/steps`"

This reverts commit 24df2d722282ff1eef60fa45cef4b0e742826d12.

* Revert "Revert "Use MLv2 in `notebook/lib/steps`""

This reverts commit 7b511fd053f59fef2624883c5136a643818e7dee.

* Fix join revert

* Fix type imports linter errors

* MLv2 Joins 8 — Update tests (#32912)

* Use new test IDs

* Update test

Join suggested condition is smarter now and we don't need to manually construct a condition here

* Fix test

* Remove irrelevant test

The behavior used in the repro is removed

* Fix test

Section names used to be inconsistent — mixing singular and plural forms

* Fix test

* Fix fails because of FK name in column names

Normally it's expected to show FK reference names here

It seems like MLv1 just couldn't resolve it through metadata before

* Fix column name

* Fix `JoinStep` tests

* Fix missing FK reference name

* Fix columns highlighting for new condition

* Fix missing FK reference name

* Update E2E tests

MLv2 now figures out suggested conditions in a better way,
and we can skip the manual selection steps now

* MLv2 Joins 9 — Remove `externalOp` usage (#33526)

* Replace `JoinConditionExternalOp` type

* Add `joinConditionParts` helper

* Revert `externalOp` method type definitions

* Don't use `externalOp` for joins

* Fix deselecting all join fields

* Fix test

* Fix tests

* Remove most of MLv1 joins code (#33334)

* Remove join's `clean` method

* Remove join's `hasGaps` method

* Remove join's strategy methods

* Remove join fields method

* Remove join's `addEmptyDimensionsPair` method

* Remove join's `joinDimensionOptions` method

* Remove join's `setJoinSourceTableId` method

* Remove join's alias methods

* Remove join's `setOperator` method

* Remove join's `removeCondition` method

* Remove join's `setDefaultCondition` method

* Remove join's `parentDimensions` method

* Remove join's dimension setter methods

* Remove join's `parentDimensionOptions` method

* Remove join's `isMultipleConditionsJoin` method

* Remove join's `joinDimensions` method

* Clean up the `Join` class

* Remove query's `clearJoins` method

* Add `getQueryStageIndex` method

* Remove query's `hasJoins` method

* Remove query's `addJoin` method

* Mark query's `joinedDimensions` as deprecated

* Mark query's `joins` as deprecated

* Mark query's `join` as deprecated

* Remove join update methods

* Remove join removal methods

* Make `cleanJoins` private

* Remove `Join.unit.spec`

* Return `clearFields` join method

* Fix wrong references

* Revert "Remove join removal methods"

This reverts commit 9ae1e8c503d9b097d073836e7c6530282c6a64fa.

* Revert "Remove join update methods"

This reverts commit de232238bd266ed72a498d0afea3675e609cfae3.

* Mark join methods as deprecated

* Remove tests

* Fix bad reference

* Fix linter error

* Revert #32911 (#34407)

* MLv2 Joins — Disable RHS table changes (#34501)

* Disable RHS table change

* Don't highlight inactive join step cells

* Use `disabled` prop instead of custom styles

* Fix test

* Fix RHS column table names

* Fix column name label

* Don't suggest joining something by its implicitly joinable columns (#34538)

Fixes #34526.

---------

Co-authored-by: metamben <103100869+metamben@users.noreply.github.com>

20d392be

20d392be 1 year ago

Name	Last commit	Last update
.babel_cache
.clj-kondo
.cljfmt
.devcontainer
.github
.husky
.lsp
.storybook
.vscode
bin
dev
docs
e2e
enterprise
frontend
hooks
locales
modules/drivers
release
resources
shared
snowplow
src/metabase
test/metabase
test_config
test_modules/drivers
test_resources
.dir-locals.el
.dockerignore
.editorconfig
.eslintignore
.eslintrc
.gitignore
.mlc_config.json
.nvmrc
.percy.yml
.pre-commit-config.yaml
.prettierignore
.prettierrc
.yarnrc
Dockerfile
LICENSE-AGPL.txt
LICENSE-EMBEDDING.txt
LICENSE-MCL.txt
LICENSE.txt
README.md
SECURITY.md
babel.config.json
codecov.yml
deps.edn
jest.config.json
jest.tz.unit.conf.json
package.json
postcss.config.js
shadow-cljs.edn
tsconfig.base.json
tsconfig.json
webpack.config.js
webpack.shared.config.js
webpack.static-viz.config.js
yarn.lock
zen.md

Metabase

Metabase is the easy, open-source way for everyone in your company to ask questions and learn from data.

Features

Set up in five minutes (we're not kidding).
Let anyone on your team ask questions without knowing SQL.
Use the SQL editor for more complex queries.
Build handsome, interactive dashboards with filters, auto-refresh, fullscreen, and custom click behavior.
Create models that clean up, annotate, and/or combine raw tables.
Define canonical segments and metrics for your team to use.
Send data to Slack or email on a schedule with dashboard subscriptions.
Set up alerts to have Metabase notify you when your data changes.
Embed charts and dashboards in your app, or even your entire Metabase.

Take a tour of Metabase.

Supported databases

Installation

Metabase can be run just about anywhere. Check out our Installation Guides.

Contributing

Quick Setup: Dev environment

In order to spin up a development environment, you need to start the front end and the backend as follows:

Frontend quick setup

The following command will install the Javascript dependencies:

$ yarn install

To build and run without watching changes:

$ yarn build

To build and run with hot-reload:

$ yarn build-hot

Backend quick setup

In order to run the backend, you'll need to build the drivers first, and then start the backend:

$ ./bin/build-drivers.sh
$ clojure -M:run

For a more detailed setup of a dev environment for Metabase, check out our Developers Guide.

Internationalization

We want Metabase to be available in as many languages as possible. See which translations are available and help contribute to internationalization using our project over at POEditor. You can also check out our policies on translations.

Extending Metabase

Hit our Query API from Javascript to integrate analytics. Metabase enables your application to:

Build moderation interfaces.
Export subsets of your users to third party marketing automation software.
Provide a custom customer lookup application for the people in your company.

Check out our guide, Working with the Metabase API.

Security Disclosure

See SECURITY.md for details.

License

This repository contains the source code for both the Open Source edition of Metabase, released under the AGPL, as well as the commercial editions of Metabase, which are released under the Metabase Commercial Software License.

See LICENSE.txt for details.

Unless otherwise noted, all files © 2023 Metabase, Inc.

Metabase Experts

If you’d like more technical resources to set up your data stack with Metabase, connect with a Metabase Expert.