Snippets Groups Projects

LLMs for Question and Dashboard Titles and Descriptions (#38298)

Mark Bastian authored 1 year ago

* Endpoint stubs

* It's alive!!!

The summarization endpoint is live, calling OpenAI, and connected to the `POST "metabot/card/summarize"`. `POST card/` also taps the results.

* Updating prompt

* Updated response shape

* [WIP] working front-end suggest button

* Incorporating viz settings

* add suggest description to save question modal

* Updating prompt

* merge title and description suggestion into one button

* Updating prompt

* Removing dead code

* Removing unused param and fn cleanup

* Fix string comment

* Adding dashboard summary endpoint stubs

* auto-request title and description

* dashboard summary wip

* dashboard summary wip

* dashboard summary wip

* better prompt

* remove dead component

* add suggest description button in dashboard sidebar

* fix dashboard description suggestion request

* better prompt

* swap out button when generating dashboard suggestion

* adding keywords

* adding keywords

* adding keywords

* adding keywords

* polish loading question name and description suggestions

* polish UI for dashboard description suggestion

* Stubbed in usage middleware

* Beginning EE Push

Moved metabot/summarize endpoints to autodescribe/summarize in the ee codebase to push this towards being an ee feature and distinguish it from metabot features.

Added snowplow tracking.

* Isolating EE LLM Code

This change completely separates metabot code from ee llm code.

It also introduces a composable client mechanism for creating LLM clients for different tasks.

* Task-Oriented LLM Refactoring

LLM code is now organized clearly into a directory of tasks. Each ns in the tasks package provides a certain LLM task to be used in the API.

* Adding unit test

* Adding admin UI elements for EE OpenAI key

* prettier on frontend/src/metabase/services.js

* Cleaning up description text

* Updating api calls

* FE stuff -- hiding things when feature not enabled.

* Figuring out linting for snowplow file

* Figuring out linting for snowplow file

* Fixing unit test

* add token feature llm-autodscription

* change added from 0.50 to 0.49

* extract logic from SaveQuestionModal into ee plugin

* extract logic from DashboardInfoSidebar to ee plugin

* Fixing unit test

* tests

* cleaning up code for SaveQuestionModal

* gate llm question name behind openAI key setting

* remove styles from SaveQuestionModal.css

* clean up code for dashboard

* lint

* Enabling triggering based on `:llm-autodescription` feature flag

* Starting to look at using a badge for LLM results (#38797)

* Starting to look at using a badge for LLM results

* WIP - Using an Indicator to show AI "stuff"

* Prototype of LLM with badge/indicator

* Removing dashboard description AI

* Ensuring LLM icon only shows when key is present. Updating icon to newer design. TODO: Put the icon in the right place.

* Cleanup -- still need to figure out how to position the indicator and not collapse the radio when selected.

* Updating save question form with LLM values does not reset form

* Cleanup

* Correct icon placement

* Adding MB branding to star

* add ai icon, tweak display

* Fix z-index, tweak padding

* cleanup

* remove inactive code

* Cleaning llm description task cobwebs out

* fix collection creation

* Cleaning lint and merge errors

* Adding "save-question-modal" test id back in

* Fixing jest test

* Touching a file to unstick CI

* Untouching a file to unstick CI

* Fixing some includes from the rebase.

* eslint fixes

* eslint fixes

* Fixing types

* Using `PLUGIN_IS_EE_BUILD.isEEBuild()` instead of `isEnterprise()`

* Removing dead styling

* Adding test ids and fixing cypress tests

* Fixing e2e tests

* Incorporating feedback

* Update frontend/src/metabase-types/api/settings.ts

Co-authored-by: Jesse Devaney <22608765+JesseSDevaney@users.noreply.github.com>

* Update frontend/src/metabase/core/components/EditableText/EditableText.tsx

Co-authored-by: Jesse Devaney <22608765+JesseSDevaney@users.noreply.github.com>

* Update frontend/src/metabase/core/components/Input/Input.tsx

Co-authored-by: Jesse Devaney <22608765+JesseSDevaney@users.noreply.github.com>

* Update frontend/src/metabase/core/components/TextArea/TextArea.styled.tsx

Co-authored-by: Jesse Devaney <22608765+JesseSDevaney@users.noreply.github.com>

* Update frontend/src/metabase/core/components/TextArea/TextArea.styled.tsx

Co-authored-by: Jesse Devaney <22608765+JesseSDevaney@users.noreply.github.com>

* Update frontend/src/metabase/dashboard/components/DashboardInfoSidebar/DashboardInfoSidebar.tsx

Co-authored-by: Jesse Devaney <22608765+JesseSDevaney@users.noreply.github.com>

* Fixing import

* Rebasing on master to fix conflicts

* Fixing types

* Revert "Update frontend/src/metabase/core/components/EditableText/EditableText.tsx"

This reverts commit fe11817086bff50f90336f30d33a1431cae662c5.

* Revert "Update frontend/src/metabase/core/components/Input/Input.tsx"

This reverts commit eef21407ee32e636a8b4e2594d502d5bbaa36870.

* Revert "Update frontend/src/metabase/core/components/TextArea/TextArea.styled.tsx"

This reverts commit dd1c22b993a6576d92547749d2eb93cb68411585.

* Revert "Update frontend/src/metabase/core/components/TextArea/TextArea.styled.tsx"

This reverts commit b169d05364e92c032ae5b6c972888a86079921e2.

* Fixing e2e tests

* address feedback

* Fixing e2e tests

* Fixing modal for models

---------

Co-authored-by: Jesse Devaney <22608765+JesseSDevaney@users.noreply.github.com>
Co-authored-by: Emmad Usmani <emmadusmani@berkeley.edu>
Co-authored-by: Kyle Doherty <5248953+kdoh@users.noreply.github.com>
Co-authored-by: Anton Kulyk <kuliks.anton@gmail.com>
Co-authored-by: Ryan Laurie <iethree@gmail.com>

f90e4db2

f90e4db2 1 year ago

Name	Last commit	Last update
.babel_cache
.clj-kondo
.devcontainer
.github
.husky
.lsp
.storybook
.vscode
bin
dev
docs
e2e
enterprise
frontend
hooks
locales
modules/drivers
release
resources
snowplow
src/metabase
test/metabase
test_config
test_modules/drivers
test_resources
.cljfmt.edn
.dir-locals.el
.dockerignore
.editorconfig
.eslintignore
.eslintrc
.gitignore
.mlc_config.json
.nvmrc
.percy.yml
.pre-commit-config.yaml
.prettierignore
.prettierrc
.yarnrc
Dockerfile
LICENSE-AGPL.txt
LICENSE-EMBEDDING.txt
LICENSE-MCL.txt
LICENSE.txt
README.md
SECURITY.md
babel.config.json
codecov.yml
deps.edn
jest.config.js
jest.tz.unit.conf.json
package.json
postcss.config.js
shadow-cljs.edn
tsconfig.base.json
tsconfig.json
webpack.config.js
webpack.shared.config.js
webpack.static-viz.config.js
yarn.lock
zen.md

Metabase

Metabase is the easy, open-source way for everyone in your company to ask questions and learn from data.

Get started

The easiest way to get started with Metabase is to sign up for a free trial of Metabase Cloud. You get support, backups, upgrades, an SMTP server, SSL certificate, SoC2 Type 2 security auditing, and more (plus your money goes toward improving Metabase). Check out our quick overview of cloud vs self-hosting. If you need to, you can always switch to self-hosting Metabase at any time (or vice versa).

Features

Set up in five minutes (we're not kidding).
Let anyone on your team ask questions without knowing SQL.
Use the SQL editor for more complex queries.
Build handsome, interactive dashboards with filters, auto-refresh, fullscreen, and custom click behavior.
Create models that clean up, annotate, and/or combine raw tables.
Define canonical segments and metrics for your team to use.
Send data to Slack or email on a schedule with dashboard subscriptions.
Set up alerts to have Metabase notify you when your data changes.
Embed charts and dashboards in your app, or even your entire Metabase.

Take a tour of Metabase.

Supported databases

Installation

Metabase can be run just about anywhere. Check out our Installation Guides.

Contributing

Quick Setup: Dev environment

In order to spin up a development environment, you need to start the front end and the backend as follows:

Frontend quick setup

The following command will install the Javascript dependencies:

$ yarn install

To build and run without watching changes:

$ yarn build

To build and run with hot-reload:

$ yarn build-hot

Backend quick setup

In order to run the backend, you'll need to build the drivers first, and then start the backend:

$ ./bin/build-drivers.sh
$ clojure -M:run

For a more detailed setup of a dev environment for Metabase, check out our Developers Guide.

Internationalization

We want Metabase to be available in as many languages as possible. See which translations are available and help contribute to internationalization using our project over at POEditor. You can also check out our policies on translations.

Extending Metabase

Hit our Query API from Javascript to integrate analytics. Metabase enables your application to:

Build moderation interfaces.
Export subsets of your users to third party marketing automation software.
Provide a custom customer lookup application for the people in your company.

Check out our guide, Working with the Metabase API.

Security Disclosure

See SECURITY.md for details.

License

This repository contains the source code for both the Open Source edition of Metabase, released under the AGPL, as well as the commercial editions of Metabase, which are released under the Metabase Commercial Software License.

See LICENSE.txt for details.

Unless otherwise noted, all files © 2024 Metabase, Inc.

Metabase Experts

If you’d like more technical resources to set up your data stack with Metabase, connect with a Metabase Expert.