Skip to content
Snippets Groups Projects
user avatar
Tim Macdonald authored
* allow formData uploads

* basic CSV uploads from collections

* add dropzone support for collections

* Add CSV uploads feature and have postgres and H2 support it (#29636)

* Add csv uploads feature and have postgres support it

* Add h2 support

* CSV: schema detection (#29650)

* CSV / Admin Upload Settings (#29709)

* allow formData uploads

* basic CSV uploads from collections

* add dropzone support for collections

* allow formData uploads

* basic CSV uploads from collections

* add dropzone support for collections

* Add CSV uploads feature and have postgres and H2 support it (#29636)

* Add csv uploads feature and have postgres support it

* Add h2 support

* Add defsettings

* Add table prefix setting

* add data upload settings

* Add newline

* update settings mock

* Whether or not uploads are enabled

---------

Co-authored-by: default avatarCal Herries <39073188+calherries@users.noreply.github.com>
Co-authored-by: default avatarCallum Herries <hi@callumherries.com>

* Create table and insert rows from CSV uploads for postgres (#29677)

* WIP create table and insert rows

* Add newline

* Use multimethods

* WIP

* Fix postgres

* Fix postgres impl

* Use detect-schema and add test

* Fix load-from-csv arglists

* Undo whitespace change

* Use empty db instead

* Add table and field tests

* Remove prn

* Uniquify names

* newline

* Add test for float

* Add a test for the table contents

* Tidy load-from-csv

* Drop the table if loading the csv fails

* Drop the table only if loading the csv fails

* whitespace

* Address SQL injection vulnerability

* Use the slugified file-name as the table-name

* Fix kondo

* Using absolute path instead of relative

* Fix kondo

* Detect types for empty CSV files appropriately (#29754)

* Create models from CSV uploads (#29966)

* Wire up CSV uploads, sync the new table

Still need to create a model

* Check database support for CSV uploads better

* Another test

* Create model after sync

* Fix request with formData

* Fix tests

* Fix tests

* Revert "Fix request with formData"

This reverts commit 66f146a41cf42feb34dbb03ed8566fa4220d0642.

* Add whitespace

---------

Co-authored-by: default avatarTim Macdonald <tim@tsmacdonald.com>

* Fix postgres support for uploads (#29922)

* Check write perms for collection (#29976)

* Rename metabase.csv -> metabase.upload (#29975)

* Rename metabase.csv -> metabase.upload

* Add docstring

* Rename alias for release.common.upload

---------

Co-authored-by: default avatarCallum Herries <hi@callumherries.com>

* WIP - 29598 csv status toast (#29757)

* temp

* generalize StatusListing types

* Putting a pin in this for now

* poorly done typing

* generalizing upload function names

* point to real API

* Fixing Storybook

* Permissions for Status Listing

* use formdata and fix types

---------

Co-authored-by: default avatarRyan Laurie <iethree@gmail.com>

* Don't support redshift (#29981)

* Handle uploads to the root collection (#29985)

* Allow the uploads schema to be nil (for MySQL, etc.) (#29987)

* CSV upload: fix stack overflow in detect-schema (#29983)

* Fix stack overflow error on detect schema

* map + vec -> mapv

---------

Co-authored-by: default avatarTim Macdonald <tim@metabase.com>

* reload collection data after upload (#30009)

* MySQL support for uploads (#29829)

* Add MySQL support for uploads

* Rename arg to upload-type

* Use new ns name

* Allow no table-name prefix to be specified and fix tests

---------

Co-authored-by: default avatarTim Macdonald <tim@tsmacdonald.com>

* Fix setting descriptions (#30023)

* Batch CSV insertion for better performance (#30028)

* CSV upload e2e tests (#30005)

* add CSV upload e2e tests

* update tests

* H2 support for uploads (#29828)

* Add uploads support for H2

* Rename arg

* whitespace

* Use new metabase.upload ns name

* Fix tests for H2

* Make table and field matching case insensitive

* Fix kondo error

---------

Co-authored-by: default avatarTim Macdonald <tim@tsmacdonald.com>

* Clear CSV upload state after a while (#30014)

* clear upload status after 5s

* make async behavior clearer

* CSV Upload Dates and Datetimes Types (#30076)

* Simple dates

* Simple datetimes

* Fix coalesce for date and datetime

* Add mysql and h2 types

* Fix test

* Misc cleanup

* Fix test

---------

Co-authored-by: default avatarTim Macdonald <tim@tsmacdonald.com>

* Fix Kondo (rebase gone wrong)

* MySQL test fix

* Fix order of Metabot setting

* adding unit tests to upload actions and reducers (#30161)

* adding unit tests to upload actions and reducers

* use fetch-mock

* Adding component unit test

* Check CSV Upload permissions on the frontend (#30094)

* CSV / Admin Upload Settings (#29709)

* allow formData uploads

* basic CSV uploads from collections

* add dropzone support for collections

* allow formData uploads

* basic CSV uploads from collections

* add dropzone support for collections

* Add CSV uploads feature and have postgres and H2 support it (#29636)

* Add csv uploads feature and have postgres support it

* Add h2 support

* Add defsettings

* Add table prefix setting

* add data upload settings

* Add newline

* update settings mock

* Whether or not uploads are enabled

---------

Co-authored-by: default avatarCal Herries <39073188+calherries@users.noreply.github.com>
Co-authored-by: default avatarCallum Herries <hi@callumherries.com>

* properly check if user can upload to a collection

* do some db checking

* don't check admin status for upload notification

* misc cleanup

* fix rebase

* add loggedin check to statusListing

also combines container + view component into a single file

* update unit tests

* update tests

---------

Co-authored-by: default avatarCal Herries <39073188+calherries@users.noreply.github.com>
Co-authored-by: default avatarCallum Herries <hi@callumherries.com>

* Csv/misc bugfixes (#30219)

* Support empty header names

* Handle long CSV names more appropriately

Handle duplicate column names in CSVs

* Test fix

* Partition CSV inserts a little more intelligently

* Test fix

* Remove 'duplicate' from duplicated column names

* More intelligent sampling

* Add comment re. Postgres table-name-length-limit

* Quote column names, clean up tests

When unquoted, we couldn't use reserved words (true, false, group, etc.)
as column names

* Fix test

Now that we quote things, H2 is case-sensitive

* Use dialect-appropriate quoting

Also clean up some card-filtering tests so they work with a dirty DB

* Fix deduplication

* Remove use of setting/get

* Remove unneeded migration

---------

Co-authored-by: default avatarCallum Herries <hi@callumherries.com>

* Fix test (#30383)

---------

Co-authored-by: default avatarRyan Laurie <iethree@gmail.com>
Co-authored-by: default avatarCal Herries <39073188+calherries@users.noreply.github.com>
Co-authored-by: default avatarRyan Laurie <30528226+iethree@users.noreply.github.com>
Co-authored-by: default avatarCallum Herries <hi@callumherries.com>
Co-authored-by: default avatarNick Fitzpatrick <nick@metabase.com>
a06e640e
History

Metabase

Metabase is the easy, open-source way for everyone in your company to ask questions and learn from data.

Metabase Product Screenshot

Latest Release Circle CI codecov Docker Pulls

Features

Take a tour of Metabase.

Supported databases

Installation

Metabase can be run just about anywhere. Check out our Installation Guides.

Contributing

To get started with a development installation of the Metabase, check out our Developers Guide.

Internationalization

We want Metabase to be available in as many languages as possible. See which translations are available and help contribute to internationalization using our project over at POEditor. You can also check out our policies on translations.

Extending Metabase

Hit our Query API from Javascript to integrate analytics. Metabase enables your application to:

  • Build moderation interfaces.
  • Export subsets of your users to third party marketing automation software.
  • Provide a custom customer lookup application for the people in your company.

Check out our guide, Working with the Metabase API.

Security Disclosure

See SECURITY.md for details.

License

This repository contains the source code for both the Open Source edition of Metabase, released under the AGPL, as well as the commercial editions of Metabase, which are released under the Metabase Commercial Software License.

See LICENSE.txt for details.

Unless otherwise noted, all files © 2023 Metabase, Inc.

Metabase Experts

If you’d like more technical resources to set up your data stack with Metabase, connect with a Metabase Expert.