Merge branch 'master' into useful-questions

bin/osx-release

-#! /usr/bin/perl -I./bin
+#! /usr/bin/env perl -I./bin
 
 use strict;
 use warnings;

bin/osx-setup

-#! /usr/bin/perl -I./bin
+#! /usr/bin/env perl -I./bin
 
 use strict;
 use warnings;

docs/administration-guide/start.md

@@ -2,15 +2,16 @@
 
 Are you in charge of managing Metabase for your organization? Then you're in the right spot. You are the chosen one.
 
-**This guide will teach you how to:**
+**This guide will teach you about:**
 
-* [Connect Metabase to databases in your organization](01-managing-databases.md)
-* [Enable features that send email (SMTP)](02-setting-up-email.md)
-* [Edit your database metadata](03-metadata-editing.md)
-* [Manage user accounts](04-managing-users.md)
+* [Connecting Metabase to databases in your organization](01-managing-databases.md)
+* [Enabling features that send email (SMTP)](02-setting-up-email.md)
+* [Editing your database metadata](03-metadata-editing.md)
+* [Managing user accounts](04-managing-users.md)
 * [Creating segments and metrics](05-segments-and-metrics.md)
-* [Configure settings](06-configuration-settings.md)
-* [Setting up Slack Integration](07-setting-up-slack.md)
+* [Configuring settings](06-configuration-settings.md)
+* [Setting up Slack integration](07-setting-up-slack.md)
+* [Enabling single sign-on with Google](08-single-sign-on.md)
 
 First things first, you'll need to install Metabase. If you haven’t done that yet, our [Installation Guide](../operations-guide/start.md#installing-and-running-metabase) will help you through the process.
 

docs/developers-guide-osx.md

@@ -6,17 +6,41 @@
 
 2.  Run `./bin/build` to build the latest version of the uberjar.
 
-3.  Next, you'll need to run the following commands before building the app:
+3.  Update Perl. I'm not sure these steps are actually needed, so feel free to try skipping it and come back to it if it fails:
+  
+   ```bash
+   # Upgrade Perl
+   brew install perl
+   
+   # Add new version of perl to your $PATH
+   # (replace "5.24.0_1" below with whatever version you installed)
+   echo 'export PATH="/usr/local/Cellar/perl/5.24.0_1/bin:$PATH"' >> ~/.bash_profile
+   source ~/.bash_profile
+   
+   # Double-check that we're using the newer version of CPAN
+   # (If this is your first time running CPAN, use the default config settings when prompted)
+   cpan --version # You should see a line like "running under Perl version 5.24.0."
+   ```
+
+4.  Next, you'll need to run the following commands before building the app:
 
     ```bash
    # Fetch and initialize git submodule
    git submodule update --init
    
-   # Install libcurl (needed by WWW::Curl::Simple)
+   # Install libcurl (needed by WWW::Curl::Simple (I think))
    brew install curl && brew link curl --force
    
-   # Install Perl modules used by ./setup and ./release
-   sudo cpan install File::Copy::Recursive JSON Readonly String::Util Text::Caml WWW::Curl::Simple
+   # The new version of LLVM is snippy so have CPAN pass compiler flags to fix errors
+   # (Make sure this file exists first. If you didn't upgrade Perl in the step above, 
+   # it might be in a different location; perhaps called "Config.pm". 
+   # You may need to run "cpan" (no arguments) to generate an appropriate initial config. 
+   # As above, you can go with the defaults).
+   sed -i '' -e "s/'make_arg' => q\[\]/'make_arg' => q\[CCFLAGS=\"-Wno-return-type\"\]/" ~/.cpan/CPAN/MyConfig.pm
+
+   # Install Perl modules used by ./bin/osx-setup and ./bin/osx-release
+   # You may have to run this as sudo if you didn't upgrade perl as described in step above
+   cpan install File::Copy::Recursive JSON Readonly String::Util Text::Caml WWW::Curl::Simple
    
    # Copy JRE and uberjar
    ./bin/osx-setup
@@ -25,17 +49,22 @@
 `./bin/osx-setup` will build run commands to build the uberjar for you if needed.
 Run `./bin/osx-setup` again at any time in the future to copy the latest version of the uberjar into the project.
 
+(If the script fails near the end, you can just copy the JARs in question to `OSX/Resources/metabase.jar` and `OSX/Resources/reset-password.jar`.)
 
 ## Releasing
 
 A handy Perl script called `./bin/osx-release` takes care of all of the details for you. Before you run it for the first time, you'll need to set up a few additional things:
 
 ```bash
+# Install aws command-line client (if needed)
+brew install awscli
+
 # Configure AWS Credentials
 # You'll need credentials that give you permission to write the metabase-osx-releases S3 bucket.
+# You just need the access key ID and secret key; use the defaults for locale and other options.
 aws configure --profile metabase
 
-# Copy & Edit Config file
+# Copy & Edit Config file. Alternative ask Cam for a copy of his
 cp bin/config.json.template bin/config.json
 emacs bin/config.json
 
@@ -44,7 +73,7 @@ emacs bin/config.json
 cp /path/to/private/key.pem OSX/dsa_priv.pem
 ```
 
-You'll probably also want an Apple Developer ID Application Certificate in your computer's keychain (ask Cam).
+You'll probably also want an Apple Developer ID Application Certificate in your computer's keychain. You'll need to generate a Certificate Signing Request from Keychain Access, and have Sameer go to [the Apple Developer Site](https://developer.apple.com/account/mac/certificate/) and generate one for you, then load the file on your computer. 
 
 After that, you are good to go:
 ```bash

docs/faq.md

@@ -36,6 +36,7 @@ We're constantly trying to walk the line between putting more functionality into
 ### Does Metabase support database X?
 
 Metabase currently supports:
+
 * Amazon Redshift
 * BigQuery
 * Crate (version 0.55 or higher)

docs/operations-guide/start.md

@@ -144,15 +144,15 @@ Metabase provides a custom migration command for upgrading H2 application databa
 1. Shutdown your Metabase instance so that it's not running.  This ensures no accidental data gets written to the db while migrating.
 2. Make a backup copy of your H2 application database by following the instructions in [Backing up Metabase Application Data](#backing-up-metabase-application-data).  Safety first!
 3. Run the Metabase data migration command using the appropriate environment variables for the target database you want to migrate to.  You can find details about specifying MySQL and Postgres databases at [Configuring the application database](#configuring-the-metabase-application-database).  Here's an example of migrating to Postgres.
-
-    export MB_DB_TYPE=postgres  
-    export MB_DB_DBNAME=metabase  
-    export MB_DB_PORT=5432  
-    export MB_DB_USER=<username>  
-    export MB_DB_PASS=<password>  
-    export MB_DB_HOST=localhost  
-    java -jar metabase.jar load-from-h2 <path-to-metabase-h2-database-file>
-
+```
+export MB_DB_TYPE=postgres
+export MB_DB_DBNAME=metabase
+export MB_DB_PORT=5432
+export MB_DB_USER=<username>
+export MB_DB_PASS=<password>
+export MB_DB_HOST=localhost
+java -jar metabase.jar load-from-h2 <path-to-metabase-h2-database-file>
+```
 It is expected that you will run the command against a brand new (empty!) database and Metabase will handle all of the work of creating the database schema and migrating the data for you.
 
 **Note:** It is required that wherever you are running this migration command can connect to the target MySQL or Postgres database.  So if you are attempting to move the data to a cloud database make sure you take that into consideration.

docs/users-guide/start.md

-#User Guide
+# User Guide
 
 **This guide will teach you:**
 

frontend/src/metabase/admin/settings/components/SettingsSetting.jsx

@@ -56,7 +56,8 @@ export default class SettingsSetting extends Component {
     }
 
     renderToggleInput(setting) {
-        const on = (setting.value == null ? setting.default : setting.value) === true;
+        const value = setting.value == null ? setting.default : setting.value,
+              on    = value === true || value === "true";
         return (
             <div className="flex align-center pt1">
                 <Toggle value={on} onChange={!this.props.disabled ? this.props.updateSetting.bind(null, setting, on ? "false" : "true") : null}/>

frontend/src/metabase/dashboard/dashboard.js

@@ -8,7 +8,7 @@ import { normalize, Schema, arrayOf } from "normalizr";
 
 import MetabaseAnalytics from "metabase/lib/analytics";
 import { getPositionForNewDashCard } from "metabase/lib/dashboard_grid";
-
+import { applyParameters } from "metabase/meta/Card";
 import { fetchDatabaseMetadata } from "metabase/redux/metadata";
 
 const DATASET_SLOW_TIMEOUT   = 15 * 1000;
@@ -143,24 +143,7 @@ export const fetchCardData = createThunkAction(FETCH_CARD_DATA, function(card, d
 
         let dashboard = dashboards[dashboardId];
 
-        let parameters = [];
-        if (dashboard && dashboard.parameters) {
-            for (const parameter of dashboard.parameters) {
-                let mapping = _.findWhere(dashcard && dashcard.parameter_mappings, { card_id: card.id, parameter_id: parameter.id });
-                if (parameterValues[parameter.id] != null) {
-                    parameters.push({
-                        type: parameter.type,
-                        target: mapping && mapping.target,
-                        value: parameterValues[parameter.id]
-                    });
-                }
-            }
-        }
-
-        let datasetQuery = {
-            ...card.dataset_query,
-            parameters
-        };
+        const datasetQuery = applyParameters(card, dashboard.parameters, parameterValues, dashcard && dashcard.parameter_mappings);
 
         let slowCardTimer = setTimeout(() => {
             if (result === null) {

frontend/src/metabase/dashboard/selectors.js

@@ -27,9 +27,8 @@ export const getParameterValues   = state => state.dashboard.parameterValues;
 export const getDatabases         = state => state.metadata.databases;
 
 export const getMetadata = createSelector(
-    [getDatabases],
-    (databases) =>
-        new Metadata(Object.entries(databases).map(([k,v]) => v)) // not sure why flow doesn't like Object.values() here
+    [state => state.metadata],
+    (metadata) => Metadata.fromEntities(metadata)
 )
 
 export const getDashboard = createSelector(

frontend/src/metabase/meta/Card.js

 /* @flow */
 
 import type { StructuredQueryObject, NativeQueryObject, TemplateTag } from "./types/Query";
-import type { CardObject, StructuredDatasetQueryObject, NativeDatasetQueryObject } from "./types/Card";
+import type { CardObject, DatasetQueryObject, StructuredDatasetQueryObject, NativeDatasetQueryObject } from "./types/Card";
+import type { ParameterObject, ParameterId, ParameterMappingObject } from "metabase/meta/types/Dashboard";
 
 declare class Object {
     static values<T>(object: { [key:string]: T }): Array<T>;
 }
 
 import * as Query from "./Query";
+import QueryLib from "metabase/lib/query";
+import _ from "underscore";
 
 export const STRUCTURED_QUERY_TEMPLATE: StructuredDatasetQueryObject = {
     type: "query",
@@ -62,3 +65,46 @@ export function getTemplateTags(card: ?CardObject): Array<TemplateTag> {
         Object.values(card.dataset_query.native.template_tags) :
         [];
 }
+
+export function applyParameters(
+    card: CardObject,
+    parameters: Array<ParameterObject>,
+    parameterValues: { [key: ParameterId]: string } = {},
+    parameterMappings: Array<ParameterMappingObject> = []
+): DatasetQueryObject {
+    const datasetQuery = JSON.parse(JSON.stringify(card.dataset_query));
+    // clean the query
+    if (datasetQuery.type === "query") {
+        datasetQuery.query = QueryLib.cleanQuery(datasetQuery.query);
+    }
+    datasetQuery.parameters = [];
+    for (const parameter of parameters || []) {
+        let value = parameterValues[parameter.id];
+
+        // dashboards
+        const mapping = _.findWhere(parameterMappings, { card_id: card.id, parameter_id: parameter.id });
+        if (value != null && mapping) {
+            datasetQuery.parameters.push({
+                type: parameter.type,
+                target: mapping.target,
+                value: value
+            });
+        }
+
+        // SQL parameters
+        if (datasetQuery.type === "native") {
+            let tag = _.findWhere(datasetQuery.native.template_tags, { id: parameter.id });
+            if (tag) {
+                datasetQuery.parameters.push({
+                    type: parameter.type,
+                    target: tag.type === "dimension" ?
+                        ["dimension", ["template-tag", tag.name]]:
+                        ["variable", ["template-tag", tag.name]],
+                    value: value
+                });
+            }
+        }
+    }
+
+    return datasetQuery;
+}

frontend/src/metabase/meta/Dashboard.js

@@ -184,7 +184,7 @@ function fieldFilterForParameter(parameter: ParameterObject): FieldFilter {
     switch (type) {
         case "date":        return (field: Field) => field.isDate();
         case "location":    return (field: Field) => field.special_type === subtype;
-        case "id":          return (field: Field) => console.log("isID", field.isID())||field.isID();
+        case "id":          return (field: Field) => field.isID();
         case "category":    return (field: Field) => field.isCategory();
     }
     return (field: Field) => false;

frontend/src/metabase/meta/metadata/Database.js

@@ -4,7 +4,7 @@ import Base from "./Base";
 import Table from "./Table";
 
 export default class Database extends Base {
-    static type = "database";
+    static type = "databases";
     static schema = {
         tables: [Table]
     };

frontend/src/metabase/meta/metadata/Field.js

@@ -6,7 +6,7 @@ import Table from "./Table";
 import { isDate, isNumeric, isBoolean, isString, isSummable, isCategory, isDimension, isMetric, getIconForField } from "metabase/lib/schema_metadata";
 
 export default class Field extends Base {
-    static type = "field";
+    static type = "fields";
     static schema = {};
 
     id: number;

frontend/src/metabase/meta/metadata/Metadata.js

@@ -9,6 +9,13 @@ export default class Metadata extends Base {
     static type = "metadata";
     static schema = { databases: [Database] };
 
+    static fromEntities(entities) {
+        const m = new Metadata([]);
+        m._entityMaps = entities;
+        m._object = { databases: Object.keys(entities.databases) };
+        return m;
+    }
+
     constructor(databases: Array<Object>) {
         super({ databases, id: 0 });
     }

frontend/src/metabase/meta/metadata/Table.js

@@ -6,7 +6,7 @@ import Database from "./Database";
 
 
 export default class Table extends Base {
-    static type = "table";
+    static type = "tables";
     static schema = {
         fields: [Field]
     };

frontend/src/metabase/meta/types/Card.js

@@ -2,6 +2,7 @@
 
 import type { DatabaseId } from "./base";
 import type { StructuredQueryObject, NativeQueryObject } from "./Query";
+import type { ParameterInstance } from "./Dashboard";
 
 export type CardId = number;
 
@@ -17,13 +18,15 @@ export type CardObject = {
 export type StructuredDatasetQueryObject = {
     type: "query",
     database: ?DatabaseId,
-    query: StructuredQueryObject
+    query: StructuredQueryObject,
+    parameters?: Array<ParameterInstance>
 };
 
 export type NativeDatasetQueryObject = {
     type: "native",
     database: ?DatabaseId,
     native: NativeQueryObject,
+    parameters?: Array<ParameterInstance>
 };
 
 export type DatasetQueryObject = StructuredDatasetQueryObject | NativeDatasetQueryObject;

frontend/src/metabase/meta/types/Dashboard.js

@@ -52,3 +52,9 @@ export type ParameterOption = {
     description?: string,
     type: ParameterType
 };
+
+export type ParameterInstance = {
+    type: ParameterType,
+    target: ParameterMappingTarget,
+    value: string
+};

frontend/src/metabase/query_builder/QueryVisualization.jsx

@@ -142,7 +142,7 @@ export default class QueryVisualization extends Component {
 
     onDownloadCSV() {
         const form = ReactDOM.findDOMNode(this._downloadCsvForm);
-        form.query.value = JSON.stringify(this.props.card.dataset_query);
+        form.query.value = JSON.stringify(this.props.fullDatasetQuery);
         form.submit();
     }
 

frontend/src/metabase/query_builder/actions.js

@@ -15,6 +15,7 @@ import Query from "metabase/lib/query";
 import { createQuery } from "metabase/lib/query";
 import { loadTableAndForeignKeys } from "metabase/lib/table";
 import Utils from "metabase/lib/utils";
+import { applyParameters } from "metabase/meta/Card";
 
 import { getParameters } from "./selectors";
 
@@ -377,13 +378,10 @@ export const updateTemplateTag = createThunkAction(UPDATE_TEMPLATE_TAG, (templat
 });
 
 export const SET_PARAMETER_VALUE = "SET_PARAMETER_VALUE";
-export const setParameterValue = createThunkAction(SET_PARAMETER_VALUE, (parameterId, value) => {
-    return (dispatch, getState) => {
-        return { id: parameterId, value };
-    };
+export const setParameterValue = createAction(SET_PARAMETER_VALUE, (parameterId, value) => {
+    return { id: parameterId, value };
 });
 
-
 export const NOTIFY_CARD_CREATED = "NOTIFY_CARD_CREATED";
 export const notifyCardCreatedFn = createThunkAction(NOTIFY_CARD_CREATED, (card) => {
     return (dispatch, getState) => {
@@ -742,40 +740,24 @@ export const setQuerySort = createThunkAction(SET_QUERY_SORT, (column) => {
     };
 });
 
+
 // runQuery
 export const RUN_QUERY = "RUN_QUERY";
-export const runQuery = createThunkAction(RUN_QUERY, (card, shouldUpdateUrl=true, paramValues) => {
+export const runQuery = createThunkAction(RUN_QUERY, (card, shouldUpdateUrl = true, parameterValues) => {
     return async (dispatch, getState) => {
         const state = getState();
         const parameters = getParameters(state);
 
         // if we got a query directly on the action call then use it, otherwise take whatever is in our current state
         card = card || state.qb.card;
-        card = JSON.parse(JSON.stringify(card));
-        let dataset_query = card.dataset_query,
-            cardIsDirty = isCardDirty(card, state.qb.originalCard);
+        parameterValues = parameterValues || state.qb.parameterValues || {};
 
-        if (dataset_query.query) {
-            // TODO: this needs to be immutable
-            dataset_query.query = Query.cleanQuery(dataset_query.query);
-        }
+        const cardIsDirty = isCardDirty(card, state.qb.originalCard);
 
-        // apply any pseudo-parameters, if specified
-        if (parameters && parameters.length > 0) {
-            let templateTags = card.dataset_query.native.template_tags || {};
-            let parameterValues = paramValues || state.qb.parameterValues || {};
-            dataset_query.parameters = parameters.map(parameter => {
-                let tag = _.findWhere(templateTags, { id: parameter.id });
-                let value = parameterValues[parameter.id];
-                if (value != null && tag) {
-                    return {
-                        type: parameter.type,
-                        target: [tag.type === "dimension" ? "dimension" : "variable", ["template-tag", tag.name]],
-                        value: value
-                    };
-                }
-            }).filter(p => p);
-        }
+        card = {
+            ...card,
+            dataset_query: applyParameters(card, parameters, parameterValues)
+        };
 
         if (shouldUpdateUrl) {
             dispatch(updateUrl(card, cardIsDirty));
@@ -785,14 +767,14 @@ export const runQuery = createThunkAction(RUN_QUERY, (card, shouldUpdateUrl=true
         let startTime = new Date();
 
         // make our api call
-        Metabase.dataset({ timeout: cancelQueryDeferred.promise }, dataset_query, function (queryResult) {
+        Metabase.dataset({ timeout: cancelQueryDeferred.promise }, card.dataset_query, function (queryResult) {
             dispatch(queryCompleted(card, queryResult));
 
         }, function (error) {
             dispatch(queryErrored(startTime, error));
         });
 
-        MetabaseAnalytics.trackEvent("QueryBuilder", "Run Query", dataset_query.type);
+        MetabaseAnalytics.trackEvent("QueryBuilder", "Run Query", card.dataset_query.type);
 
         // HACK: prevent SQL editor from losing focus
         try { ace.edit("id_sql").focus() } catch (e) {}