Merge branch 'master' into merge-master-again

docs/administration-guide/databases/bigquery.md

@@ -18,11 +18,11 @@ To create the service account JSON file, follow Google's documentation on [setti
 
 3. **Grant the service account access to this project**. You'll need to add **roles** to the service account so that Metabase will have permission to view and run queries against your dataset. Make sure you add the following roles to the service account:
 
-    - BigQuery Data Viewer
-    - BigQuery Metadata Viewer
-    - BigQuery Job User (distinct from BigQuery User)
+   - BigQuery Data Viewer
+   - BigQuery Metadata Viewer
+   - BigQuery Job User (distinct from BigQuery User)
 
-    For more information on **roles** in BigQuery, see [Google Cloud Platform's documentation](https://cloud.google.com/bigquery/docs/access-control).
+   For more information on **roles** in BigQuery, see [Google Cloud Platform's documentation](https://cloud.google.com/bigquery/docs/access-control).
 
 4. **Create key**. Once you have assigned roles to the service account, click on the **Create Key** button, and select **JSON** for the **key type**. The JSON file will download to your computer.
 
@@ -30,7 +30,7 @@ To create the service account JSON file, follow Google's documentation on [setti
 
 ## Metabase: adding a BigQuery dataset
 
-Once you have created and downloaded your service account JSON file for your BigQuery dataset, head over to your Metabase instance, click on the **settings** cog, and select **Admin** to bring up Admin mode. In the **Databases** section, click on the **Add database** button in the upper right. 
+Once you have created and downloaded your service account JSON file for your BigQuery dataset, head over to your Metabase instance, click on the **settings** cog, and select **Admin** to bring up Admin mode. In the **Databases** section, click on the **Add database** button in the upper right.
 
 On the **ADD DATABASE** page, select **BigQuery** from the **Database type** dropdown. Metabase will present you with the relevant configuration settings to fill out:
 
@@ -50,7 +50,7 @@ Each BigQuery dataset will have a **Dataset ID**. You can find this ID via the [
 
 #### Service account JSON file
 
-Upload the service account JSON file you created when following the [steps above](#google-cloud-platform:-creating-a-service-account-and-json-file). The JSON file contains the credentials your Metabase application will need to read and query your dataset, as defined by the **roles** you added to the service account. If you need to add additional **roles**, you have to create another service account, download the JSON file, and upload the file to Metabase.
+Upload the service account JSON file you created when following the [steps above](#google-cloud-platform-creating-a-service-account-and-json-file). The JSON file contains the credentials your Metabase application will need to read and query your dataset, as defined by the **roles** you added to the service account. If you need to add additional **roles**, you have to create another service account, download the JSON file, and upload the file to Metabase.
 
 #### Use the Java Virtual Machine (JVM) timezone
 

docs/administration-guide/databases/mongodb.md

@@ -12,6 +12,7 @@ This article covers:
 Go to Admin -> Databases, and click the **Add database** button. Select MongoDB from the dropdown.
 
 There are two ways to connect to MongoDB:
+
 1. Using the [Metabase fields to input your connection details](#using-metabase-fields).
 2. Pasting your [connection string](#using-a-connection-string).
 
@@ -49,19 +50,19 @@ If you need to use a certificate, connect via the [default method](#using-metaba
 ### Settings common to both connection options
 
 - **Use an SSH tunnel for database connections.** Some database installations can only be accessed by connecting through an SSH bastion host. This option also provides an extra layer of security when a VPN is not available. Enabling this is usually slower than a direct connection.
-- **Automatically run queries when doing simple filtering and summarizing.** When this is on, Metabase will automatically run queries when users do simple explorations with the Summarize and Filter buttons when viewing a table or chart. You can turn this off if querying this database is slow. This setting doesn’t affect drill-throughs or SQL queries. 
+- **Automatically run queries when doing simple filtering and summarizing.** When this is on, Metabase will automatically run queries when users do simple explorations with the Summarize and Filter buttons when viewing a table or chart. You can turn this off if querying this database is slow. This setting doesn’t affect drill-throughs or SQL queries.
 - **This is a large database, so let me choose when Metabase syncs and scans.** By default, Metabase does a lightweight hourly sync and an intensive daily scan of field values. If you have a large database, we recommend turning this on and reviewing when and how often the field value scans happen.
 
- ## Configuring SSL via the command line
- 
- You can enter a self-signed certificate via the Metabase UI (though not when using a connection string), or you can use the command line to add a self-signed certificate. 
+## Configuring SSL via the command line
+
+You can enter a self-signed certificate via the Metabase UI (though not when using a connection string), or you can use the command line to add a self-signed certificate.
 
- ```
- cp /usr/lib/jvm/default-jvm/jre/lib/security/cacerts ./cacerts.jks
- keytool -import -alias cacert -storepass changeit -keystore cacerts.jks -file my-cert.pem
-  ```
+```
+cp /usr/lib/jvm/default-jvm/jre/lib/security/cacerts ./cacerts.jks
+keytool -import -alias cacert -storepass changeit -keystore cacerts.jks -file my-cert.pem
+```
 
- Then, start Metabase using the store:
+Then, start Metabase using the store:
 
 ```
 java -Djavax.net.ssl.trustStore=cacerts.jks -Djavax.net.ssl.trustStorePassword=changeit -jar metabase.jar
@@ -71,30 +72,27 @@ Learn more about [configuring SSL with MongoDB](http://mongodb.github.io/mongo-j
 
 ## Connecting to a MongoDB Atlas cluster
 
- To make sure you are using the correct connection configuration:
+To make sure you are using the correct connection configuration:
 
-  1. Log into your [Atlas cluster](https://cloud.mongodb.com)
+1. Log into your [Atlas cluster](https://cloud.mongodb.com)
 
-  2. Select the cluster you want to connect to, and click **Connect**.
+2. Select the cluster you want to connect to, and click **Connect**.
 
-     ![Your cluster screengrab](../images/mongo_1.png "Your cluster")
+   ![Your cluster screengrab](../images/mongo_1.png "Your cluster")
 
-  3. Click **Connect Your Application**.
+3. Click **Connect Your Application**.
 
-     ![Connect screengrab](../images/mongo_2.png "Connect")
+   ![Connect screengrab](../images/mongo_2.png "Connect")
 
-  4. Select **Java** and **3.6 or later**.
+4. Select **Java** and **3.6 or later**.
 
-     ![Java screengrab](../images/mongo_3.png "Java")
+   ![Java screengrab](../images/mongo_3.png "Java")
 
-  5. The resulting connection string has the relevant information to provide to Metabase's `Add a Database` form for MongoDB.
- 
-  6. You will likely want to select the option `Use DNS SRV`, which newer Atlas clusters use by default.
+5. The resulting connection string has the relevant information to provide to Metabase's `Add a Database` form for MongoDB.
 
-## General connectivity concerns
-
- - **Connect using `DNS SRV`**, which is the recommended method for newer Atlas clusters.
-
- - **Have you checked your cluster host whitelist?** When testing a connection but seeing failure, have you tried setting the IP whitelist to `0.0.0.0/0`?  Whitelisting this address allows connections from any IP addresses. If you know the IP address(es) or CIDR block of clients, use that instead.
+6. You will likely want to select the option `Use DNS SRV`, which newer Atlas clusters use by default.
 
+## General connectivity concerns
 
+- **Connect using `DNS SRV`**, which is the recommended method for newer Atlas clusters.
+- **Have you checked your cluster host whitelist?** When testing a connection but seeing failure, have you tried setting the IP whitelist to `0.0.0.0/0`? Whitelisting this address allows connections from any IP addresses. If you know the IP address(es) or CIDR block of clients, use that instead.

docs/data-warehousing.md

@@ -20,7 +20,7 @@ Assuming you do not need to do a lot of transformation or ingest lots of third p
 
 ## Dedicated analytics database
 
-Typically once enough data is in the system and/or the tranformation needs are complex enough, a dedicated analytics database is used. There are many options ranging from a normal general purpose database (MySQL, Postgres, SQL Server, etc), to a dedicated Analytics database (Vertica, Redshift, GreenPlum, Terredata, etc), the new generation of SQL on Hadoop databases (Spark, Presto) or NoSQL databases (Druid, Cassandra, etc). 
+Typically once enough data is in the system and/or the transformation needs are complex enough, a dedicated analytics database is used. There are many options ranging from a normal general purpose database (MySQL, Postgres, SQL Server, etc), to a dedicated Analytics database (Vertica, Redshift, GreenPlum, Terredata, etc), the new generation of SQL on Hadoop databases (Spark, Presto) or NoSQL databases (Druid, Cassandra, etc). 
 
 
 It is rare that your applications database will have all the data you need and be structured in a way that lets you ask all of the questions you are interested in. Typically an application database will have a schema optimized for small reads and updates, while most analytics queries typically touch a large fraction of a table. 

docs/developers-guide-osx.md

@@ -4,38 +4,53 @@ NOTE: These instructions are only for packaging a built Metabase uberjar into `M
 our [developers' guide](developers-guide.md).
 
 ## First-Time Configuration
-
+<details>
+<summary>
+Steps
+</summary>
+   
 ### Building
 
 The following steps need to be done before building the Mac App:
 
-1.  Install XCode.
+1. Install XCode.
 
-1.  Add a JRE to the `OSX/Metabase/jre`
+1. Add a JRE to the `/path/to/metabase/repo/OSX/Metabase/jre`
+   
+   You must acquire a copy of a JRE (make sure you get a JRE rather than JDK) and move it to the correct location in the Mac App source directory so it can be included as part of the Mac App. To ship Java applications as Mac Apps, you must ship them with their own JRE. In this case we want to get a JRE from somewhere (more on this below) and move the `Contents/Home` directory from the JRE archive into `OSX/Metabase/jre`. (`OSX/Metabase` already exists inside the `metabase/metabase` repo.)
 
+   <details><summary>Option 1: Download from AdoptOpenJDK (currently broken -- do not use)</summary>
+   
     You can download a copy of a JRE from https://adoptopenjdk.net/releases.html?jvmVariant=hotspot — make sure you download a JRE rather than JDK. Move the `Contents/Home` directory from the JRE archive into `OSX/Metabase/jre`. (`OSX/Metabase` already exists inside the `metabase/metabase` repo.) For example:
 
-    ```bash
-    cd /path/to/metabase/repo
-    wget https://github.com/AdoptOpenJDK/openjdk11-binaries/releases/download/jdk-11.0.8%2B10/OpenJDK11U-jre_x64_mac_hotspot_11.0.8_10.tar.gz
-    tar -xzvf OpenJDK11U-jre_x64_mac_hotspot_11.0.8_10.tar.gz
-    mv jdk-11.0.8+10-jre/Contents/Home OSX/Metabase/jre
-    ```
+   ```bash
+   # IMPORTANT -- DO NOT COPY THIS -- THIS JRE DOESN'T WORK
+   cd /path/to/metabase/repo
+   wget https://github.com/AdoptOpenJDK/openjdk11-binaries/releases/download/jdk-11.0.8%2B10/OpenJDK11U-jre_x64_mac_hotspot_11.0.8_10.tar.gz
+   tar -xzvf OpenJDK11U-jre_x64_mac_hotspot_11.0.8_10.tar.gz
+   mv jdk-11.0.8+10-jre/Contents/Home OSX/Metabase/jre
+   ```
 
-    **VERY IMPORTANT!**
+   **VERY IMPORTANT!**
 
-    Make sure the JRE version you use is one that is known to work successfully with notarization. I'm pretty sure this version will work. Note to self -- update page once we verify that this version works.
+   Make sure the JRE version you use is one that is known to work successfully with notarization. We have found out the one linked above does not work. 
+   I have found a nightly build that *does* work, but it's no longer available for download. Cam and Sameer both have copies of a JRE that is known to work. Refer to Option 2.
 
-    If you get notarization errors like
+   If you get notarization errors like
 
-    > The executable does not have the hardened runtime enabled.
+   > The executable does not have the hardened runtime enabled.
 
-    (Referring to files in `Metabase.app/Contents/Resources/jre/bin/`) then use a different build of the JRE.
+   (Referring to files in `Metabase.app/Contents/Resources/jre/bin/`) then use a different build of the JRE.
 
-    Assuming the OpenJDK folks have resolved this issue going forward, you are fine to use whatever the latest JRE version available is. I have been using the HotSpot JRE instead of the
-    OpenJ9 one but it ultimately shouldn't make a difference.
+   Assuming the OpenJDK folks have resolved this issue going forward, you are fine to use whatever the latest JRE version available is. I have been using the HotSpot JRE instead of the OpenJ9 one but it ultimately shouldn't make a difference.
+   </details>
+   
+   <details><summary>Option 2: Ask Cam or Sameer for known working JRE</summary>
+    
+    Have Cam or Sameer ZIP up their `/path/to/metabase/repo/OSX/Metabase/jre` folder and send it to you. Don't try Option 1 until we know the issues are fixed
+    </details>
 
-1)  Copy Metabase uberjar to OSX resources dir
+1. Copy Metabase uberjar to OSX resources dir
 
     ```bash
     cp /path/to/metabase.jar OSX/Resources/metabase.jar
@@ -47,9 +62,9 @@ At this point, you should try opening up the Xcode project and building the Mac
 
 ### Releasing
 
-The following steps are prereqs for releasing the Mac App:
+The following steps are prereqs for *releasing* the Mac App:
 
-1.  Install XCode command-line tools. In `Xcode` > `Preferences` > `Locations` select your current Xcode version in the `Command Line Tools` drop-down.
+1)  Install XCode command-line tools. In `Xcode` > `Preferences` > `Locations` select your current Xcode version in the `Command Line Tools` drop-down.
 
 1)  Install AWS command-line client (if needed)
 
@@ -97,7 +112,7 @@ The following steps are prereqs for releasing the Mac App:
 
 1)  Create an App-Specific password for the Apple ID in the previous step
 
-    Go to https://appleid.apple.com/account/manage then `Security` > `App-Specific Passwords` > `Generate Password`
+    1.  Go to https://appleid.apple.com/account/manage then `Security` > `App-Specific Passwords` > `Generate Password`
 
     1.  Store the password in Keychain
 
@@ -114,6 +129,7 @@ The following steps are prereqs for releasing the Mac App:
     brew install clojure
     ```
 
+</details>
 
 ## Building & Releasing the Mac App
 

docs/enterprise-guide/sql-snippets.md

@@ -12,7 +12,7 @@ The **Top folder** is the snippet sidebar's default folder. It is the root folde
 
 ### Creating a new SQL snippet folder
 
-You can create a SQL snippet folder from the **Snippets** menu in the [SQL editor](../users-guide/writing-sql).
+You can create a SQL snippet folder from the **Snippets** menu in the [SQL editor](../users-guide/writing-sql.html).
 
 ![Create new snippet folder](./images/sql-snippets/snippet-folder.png)
 
@@ -24,7 +24,7 @@ You can create a SQL snippet folder from the **Snippets** menu in the [SQL edito
 
 ### Creating a new SQL snippet
 
-When creating a SQL snippet in the Enterprise Edition, you'll also see an additional option to add that snippet to an existing folder (the **Folder this should be in** option). 
+When creating a SQL snippet in the Enterprise Edition, you'll also see an additional option to add that snippet to an existing folder (the **Folder this should be in** option).
 
 ![Add a snippet enterprise modal](./images/sql-snippets/enterprise-add-snippet.png)
 
@@ -50,7 +50,7 @@ When changing permissions on a folder that has sub-folders, you have an option t
 
 There are three options for changing snippet folder permissions:
 
-- **Edit access (green checkmark icon)**. The default setting. When a folder is created, all users (who have SQL editor permissions for at least one database) can view, edit, and archive or unarchive the folder's snippets. 
+- **Edit access (green checkmark icon)**. The default setting. When a folder is created, all users (who have SQL editor permissions for at least one database) can view, edit, and archive or unarchive the folder's snippets.
 - **View access (yellow eye icon)**. Users in groups with view access can view snippets in the folder, but not edit or archive/unarchive them. They can, of course, copy snippet code and create new snippets, with no effect on the existing snippets.
 - **Revoke access (red X icon)**. Users in groups with neither edit nor view permissions to a snippet folder will not see that folder's snippets in the sidebar, nor will any snippets in that folder appear in typeahead suggestions for those users. Note that if people have access to a question with a snippet they don't have permission to, they will still be able to get results from that question. See the discussion below on [how folder permissions work](#how-folder-permissions-work).
 
@@ -60,7 +60,7 @@ Archiving or unarchiving snippets does not affect a snippet's permissions. If, f
 
 ### How folder permissions work
 
-As with collection permissions, folder permissions are additive, meaning that the more permissive access overrules less permissive access. See [Setting permissions](../administration-guide/05-setting-permissions) for a more detailed discussion.
+As with collection permissions, folder permissions are additive, meaning that the more permissive access overrules less permissive access. See [Setting permissions](../administration-guide/05-setting-permissions.html) for a more detailed discussion.
 
 But snippet folder permissions require some more effort to unpack, as permissions for snippet folders must work in conjunction with permissions for data and collections.
 

docs/users-guide/05-visualizing-results.md

@@ -79,7 +79,7 @@ Open up the settings for your table and you'll see the Columns tab, which displa
 
 To hide a column, click the X icon on it; that'll send it down to the **More columns** area in case you want to bring it back. To add a linked column, just click the + icon on it, which will bring it to the **Visible columns** section. Click and drag any of the columns listed there to rearrange the order in which they appear. Another super easy way to rearrange columns without having to open up the visualization settings is to simply click and drag on a column's heading to move it where you'd like it to go.
 
-> Changing these options doesn't change the actual table itself; these changes create a custom view of the table that you can save as a **question** in Metabase and refer back to later, share with others, or add to a [dashboard](08-dashboards.md).
+> Changing these options doesn't change the actual table itself; these changes create a custom view of the table that you can save as a **question** in Metabase and refer back to later, share with others, or add to a [dashboard](07-dashboards.md).
 
 #### Column formatting options
 

docs/users-guide/sql-snippets.md

@@ -89,4 +89,4 @@ Note: two snippets cannot share the same name, as even if a snippet is archived,
 
 Any user who has SQL editor permissions to at least one of your connected databases will be able to view the snippets sidebar, and will be able to create, edit, and archive or unarchive any and all snippets — even snippets intended to be used with databases the user does NOT have SQL editing access to.
 
-[Metabase Enterprise Edition](https://www.metabase.com/enterprise/) contains additional functionality for organizing snippets into folders and setting permissions on those folders. See our [docs on SQL snippet folders and permissions](../enterprise-guide/sql-snippets) to learn more.
+[Metabase Enterprise Edition](https://www.metabase.com/enterprise/) contains additional functionality for organizing snippets into folders and setting permissions on those folders. See our [docs on SQL snippet folders and permissions](../enterprise-guide/sql-snippets.html) to learn more.

modules/drivers/snowflake/src/metabase/driver/snowflake.clj

@@ -225,7 +225,7 @@
         excluded-schemas (set (sql-jdbc.sync/excluded-schemas driver))]
     {:tables (set (for [table (jdbc/query
                                (sql-jdbc.conn/db->pooled-connection-spec database)
-                               (format "SHOW TABLES IN DATABASE \"%s\"" db-name))
+                               (format "SHOW OBJECTS IN DATABASE \"%s\"" db-name))
                         :when (not (contains? excluded-schemas (:schema_name table)))]
                     {:name        (:name table)
                      :schema      (:schema_name table)

modules/drivers/snowflake/test/metabase/driver/snowflake_test.clj

 (ns metabase.driver.snowflake-test
-  (:require [clojure
+  (:require [clojure.java.jdbc :as jdbc]
+            [clojure
              [set :as set]
              [string :as str]
              [test :refer :all]]
@@ -7,11 +8,19 @@
              [driver :as driver]
              [models :refer [Table]]
              [query-processor :as qp]
-             [test :as mt]]
+             [sync :as sync]
+             [test :as mt]
+             [util :as u]]
+            [metabase.driver.sql-jdbc
+             [connection :as sql-jdbc.conn]
+             [execute :as sql-jdbc.execute]]
+            [metabase.models
+             [database :refer [Database]]]
             [metabase.test.data
              [dataset-definitions :as dataset-defs]
              [sql :as sql.tx]]
-            [metabase.test.data.sql.ddl :as ddl]))
+            [metabase.test.data.sql.ddl :as ddl]
+            [toucan.db :as db]))
 
 ;;
 (deftest ddl-statements-test
@@ -67,6 +76,28 @@
           (is (= expected
                  (driver/describe-database :snowflake (assoc (mt/db) :name "ABC")))))))))
 
+(deftest describe-database-views-test
+  (mt/test-driver :snowflake
+    (testing "describe-database views"
+      (let [details (mt/dbdef->connection-details :snowflake :db {:database-name "views_test"})
+            spec    (sql-jdbc.conn/connection-details->spec :snowflake details)]
+        ;; create the snowflake DB
+        (jdbc/execute! spec ["DROP DATABASE IF EXISTS \"views_test\";"]
+                       {:transaction? false})
+        (jdbc/execute! spec ["CREATE DATABASE \"views_test\";"]
+                       {:transaction? false})
+        ;; create the DB object
+        (mt/with-temp Database [database {:engine :snowflake, :details (assoc details :db "views_test")}]
+          (let [sync! #(sync/sync-database! database)]
+            ;; create a view
+            (jdbc/execute! spec ["CREATE VIEW \"views_test\".\"PUBLIC\".\"example_view\" AS SELECT 'hello world' AS \"name\";"])
+            ;; now sync the DB
+            (sync!)
+            ;; now take a look at the Tables in the database, there should be an entry for the view
+            (is (= [{:name "example_view"}]
+                   (map (partial into {})
+                        (db/select [Table :name] :db_id (u/get-id database)))))))))))
+
 (deftest describe-table-test
   (mt/test-driver :snowflake
     (testing "make sure describe-table uses the NAME FROM DETAILS too"