Merge branch 'master' into new-dash-plus-scalable-cards-plus-fullscreen

README.md

@@ -54,7 +54,9 @@ For a more detailed walkthrough, check out our [Getting Started](docs/getting-st
 
 # Contributing
 
-To get started with a development installation of the Query Server and learn more about contributing, please follow the instructions at our [Developers Guide](docs/developers-guide.md).
+To get started with a development installation of the Metabase, follow the instructions at our [Developers Guide](docs/developers-guide.md).
+
+The take a look at our [Contribution Guide](docs/contributing.md) for information about our process and where you can fit in!
 
 # Extending and Deep Integrations
 

docs/contributing.md

+## Thank you
+
+First off, thanks for your interest in Metabase and for wanting to contribute!
+
+In this guide, we'll discuss how Metabase is built. This should give you a good sense of our process and where you might want to fit in.
+
+## What we're trying to build
+
+Metabase is all about letting non-technical users get access to the their organization's data. We're trying to maximize the amount of power that can be comfortably used by someone who understands their business, is quantitatively bent, but probably only comfortable with Excel. 
+
+It's important to keep in mind these goals of the Metabase project. Many times
+proposals will be marked "Out of Scope" or otherwise deprioritized. This doesn't mean the proposal isn't useful, or that we wouldn't be interested in seeing it done as a side project or as an experimental branch. However, it does mean that we won't point the core team or contributors to it in the near term. Issues that are slightly out of scope will be kept open in case there is community support (and ideally contributions).
+
+To get a sense for the end goals, make sure to read the [Zen of Metabase](../zen)
+
+## Our product process:
+
+The core team runs a pretty well defined product process. It is actively being tweaked, but the below is a pretty faithful description of it at the time of writing. You should have a clear idea of how we work before jumping in with a PR. 
+
+### A) Identify product needs from the community
+
+We actively look for new feature ideas from our community, user base and our own use of Metabase internally. We concentrate on the underlying *problem* or *need*  as opposed to requests for specific features. While sometimes suggested features are built as requested, often we find that they involve changes to existing features, and perhaps an entirely different solution to the underlying problem. 
+
+### B) Synthesize these needs into a concrete feature
+
+We typically will collect a group of issues or suggestions into a new topline feature concept. Typically we'll create a working document that collects all "Open Questions" regarding to what the feature is meant to do, and more importantly not do. We'll chat with our users, maybe do in depth interviews and generally try to tightly define the feature.
+
+### C) Design the feature
+
+Once a feature has been defined, typically it will be taken on by a product designer. Here, they will produce low fi mocks, get feedback from our users and community, and iterate.
+
+Once the main UX flows have been dialed in, there will be a hi-fidelity visual design.
+
+### D) Hand off the design to an engineer
+
+This design is then handed off to one or more engineers.
+Together with the designer who owns the feature, they will work on the feature via a Pull Request.
+
+### E) Verification and merging
+
+If all goes well, the feature gets coded up, verified and then the pull request gets merged! High-fives all around.
+
+## Ways to help:
+
+The starting point would be to get familiar with Metabase the product, and know your way around. If you're using it at work, that's great! If not, download the mac application or the jar and play around with it. Read the docs and generally get a feel for the flow of the product.
+
+If you want to help out, there are lots of ways. In order of increasing coordination + interaction with us:
+
+### Help with identifying needs and problems Metabase can solve
+
+If you want to help, try out Metabase. Use it at your company, and report back the things you like, dislike and any problems you run into. Help us understand your data model, required metrics and common usage patterns as much as you can. This information directly affects the quality of the product. The more you tell us about the kinds of problems you're facing, the better we'll be able to address them.
+
+### Help us triage and support other users
+
+Spend time on discourse.metabase.com and on new issues and try to reproduce the bugs reported. For people having trouble with their databases where you have significant knowledge, help them out. Who knows, maybe they'll end up helping you with something in the future.
+
+### Tell your friends
+
+Let your friends know about Metabase. Start a user group in your area. [Tweet about us] (http://twitter.com/metabase). Blog about how you're using Metabase, and share what you've learned.
+
+### Fix bugs
+
+By our definition, "Bugs" are situations where the program doesn't do what it was expected to according to the design or specification. These are typically scoped to issues where there is a clearly defined correct behaviour. It's usually safe to grab one of these, fix it, and submit a PR (with tests!). These will be merged without too much drama unless the PR touches a lot of code. Don't be offended if we ask you to make small modifications or add more tests. We're a bit OCD on code coverage and coding style.
+
+### Help with Documentation
+
+There are a lot of docs. We often have difficulties keeping them up to date. If you are reading them and you notice inconsistencies, errors or outdated information, please help up keep them current!
+
+### Working on features that don't need visual design
+
+Some features, eg Database drivers, don't have any user facing pixels. These are a great place to start off contributing as they don't require as much communication, discussions about tradeoffs and process in general. 
+
+### Working on features that already have a visual design
+
+In situations where a design has already been done, we can always use some help. Chime in on a pull request or an issue and offer to help. 
+
+### #YOLO JUST SUBMIT A PR
+
+If you come up with something really cool, and want to share it with us, just submit a PR. If it hasn't gone through the above process, we probably won't merge it as is, but if it's compelling, we're more than willing to help you via code review, design review and generally OCD nitpicking so that it fits into the rest of our codebase. 
\ No newline at end of file

frontend/src/admin/settings/components/SettingsSlackForm.jsx

@@ -3,6 +3,7 @@ import React, { Component, PropTypes } from "react";
 import MetabaseAnalytics from "metabase/lib/analytics";
 import MetabaseUtils from "metabase/lib/utils";
 import SettingsEmailFormElement from "./SettingsEmailFormElement.jsx";
+import SettingsSetting from "./SettingsSetting.jsx";
 
 import Icon from "metabase/components/Icon.jsx";
 
@@ -34,7 +35,7 @@ export default class SettingsSlackForm extends Component {
         // this gives us an opportunity to load up our formData with any existing values for elements
         let formData = {};
         this.props.elements.forEach(function(element) {
-            formData[element.key] = element.value;
+            formData[element.key] = element.value || element.defaultValue;
         });
 
         this.setState({formData});
@@ -156,10 +157,22 @@ export default class SettingsSlackForm extends Component {
             let errorMessage = (formErrors && formErrors.elements) ? formErrors.elements[element.key] : validationErrors[element.key],
                 value = formData[element.key] || element.defaultValue;
 
-            return <SettingsEmailFormElement
+            if (element.key === "slack-token") {
+                return (
+                    <SettingsEmailFormElement
                         key={element.key}
                         element={_.extend(element, {value, errorMessage })}
                         handleChangeEvent={this.handleChangeEvent.bind(this)} />
+                );
+            } else if (element.key === "metabot-enabled") {
+                return (
+                    <SettingsSetting
+                        key={element.key}
+                        setting={_.extend(element, {value, errorMessage })}
+                        updateSetting={(setting, value) => this.handleChangeEvent(setting, value)}
+                    />
+                );
+            }
         });
 
         let saveSettingsButtonStates = {

frontend/src/admin/settings/settings.controllers.js

@@ -105,7 +105,15 @@ const SECTIONS = [
                 type: "string",
                 required: true,
                 autoFocus: true
-            }
+            },
+            {
+                key: "metabot-enabled",
+                display_name: "Metabot",
+                type: "boolean",
+                defaultValue: "true",
+                required: true,
+                autoFocus: false
+            },
         ]
     }
 ];

frontend/src/query_builder/DataReference.jsx

@@ -83,7 +83,7 @@ export default class DataReference extends Component {
         );
 
         return (
-            <div className="DataReference-container p3 scroll-y">
+            <div className="DataReference-container p3 full-height scroll-y">
                 <div className="DataReference-header flex mb1">
                     {backButton}
                     {closeButton}

src/metabase/api/slack.clj

@@ -3,18 +3,20 @@
   (:require [compojure.core :refer [PUT]]
             [metabase.api.common :refer :all]
             [metabase.config :as config]
-            [metabase.integrations.slack :as slack]))
+            [metabase.integrations.slack :as slack]
+            [metabase.models.setting :as setting]))
 
 (defendpoint PUT "/settings"
-  "Update the `slack-token`. You must be a superuser to do this."
-  [:as {{slack-token :slack-token} :body}]
-  {slack-token [Required NonEmptyString]}
+  "Update Slack related settings. You must be a superuser to do this."
+  [:as {{slack-token :slack-token, metabot-enabled :metabot-enabled, :as slack-settings} :body}]
+  {slack-token     [Required NonEmptyString]
+   metabot-enabled [Required]}
   (check-superuser)
   (try
-    ;; just check that channels.list doesn't throw an exception (that the connection works)
+    ;; just check that channels.list doesn't throw an exception (a.k.a. that the token works)
     (when-not config/is-test?
       (slack/GET :channels.list, :exclude_archived 1, :token slack-token))
-    (slack/slack-token slack-token)
+    (setting/set-all slack-settings)
     {:ok true}
     (catch clojure.lang.ExceptionInfo info
       {:status 400, :body (ex-data info)})))

src/metabase/events/metabot_lifecycle.clj

+(ns metabase.events.metabot-lifecycle
+  (:require [clojure.core.async :as async]
+            [clojure.tools.logging :as log]
+            [metabase.db :as db]
+            [metabase.driver :as driver]
+            [metabase.events :as events]
+            [metabase.metabot :as metabot]
+            [metabase.models.database :refer [Database]]))
+
+
+(def ^:const ^:private metabot-lifecycle-topics
+  "The `Set` of event topics which are subscribed to for use in metabot lifecycle."
+  #{:settings-update})
+
+(def ^:private metabot-lifecycle-channel
+  "Channel for receiving event notifications we want to subscribe to for metabot lifecycle events."
+  (async/chan))
+
+
+;;; ## ---------------------------------------- EVENT PROCESSING ----------------------------------------
+
+
+(defn process-metabot-lifecycle-event
+  "Handle processing for a single event notification received on the metabot-lifecycle-channel"
+  [metabot-lifecycle-event]
+  ;; try/catch here to prevent individual topic processing exceptions from bubbling up.  better to handle them here.
+  (when-let [{topic :topic object :item} metabot-lifecycle-event]
+    (try
+      ;; if someone updated our slack-token, or metabot was enabled/disabled then react accordingly
+      (let [{:keys [slack-token metabot-enabled]} object]
+        (cond
+          (and (contains? object :metabot-enabled)
+               (not (= "true" metabot-enabled)))   (metabot/stop-metabot!)
+          (and (contains? object :slack-token)
+               (seq slack-token))                  (metabot/start-metabot!)))
+      (catch Throwable e
+        (log/warn (format "Failed to process driver notifications event. %s" topic) e)))))
+
+
+
+;;; ## ---------------------------------------- LIFECYLE ----------------------------------------
+
+
+(defn events-init
+  "Automatically called during startup; start event listener for metabot lifecycle events."
+  []
+  (events/start-event-listener metabot-lifecycle-topics metabot-lifecycle-channel process-metabot-lifecycle-event))

src/metabase/integrations/slack.clj

@@ -7,13 +7,7 @@
 
 
 ;; Define a setting which captures our Slack api token
-(defsetting slack-token "Slack API bearer token obtained from https://api.slack.com/web#authentication" nil
-  :setter (fn [new-value]
-            (setting/set* :slack-token new-value)
-            (require 'metabase.metabot)
-            ((ns-resolve 'metabase.metabot (if (seq new-value)
-                                              'start-metabot!
-                                              'stop-metabot!)))))
+(defsetting slack-token "Slack API bearer token obtained from https://api.slack.com/web#authentication" nil)
 
 (def ^:private ^:const ^String slack-api-base-url "https://slack.com/api")
 (def ^:private ^:const ^String files-channel-name "metabase_files")

src/metabase/metabot.clj

@@ -13,10 +13,12 @@
             [metabase.api.common :refer [let-404]]
             [metabase.db :refer [sel]]
             [metabase.integrations.slack :as slack]
+            [metabase.models.setting :as setting]
             [metabase.task.send-pulses :as pulses]
             [metabase.util :as u]
             [metabase.util.urls :as urls]))
 
+(setting/defsetting metabot-enabled "Enable Metabot, which lets you search for and view your saved questions directly via Slack." "true")
 
 ;;; # ------------------------------------------------------------ Metabot Command Handlers ------------------------------------------------------------
 
@@ -37,7 +39,6 @@
                           {(if (true? dispatch-token)
                              (keyword symb)
                              dispatch-token) varr}))]
-    (println (u/format-color 'cyan verb) fn-map)
     (fn dispatch*
       ([]
        (keys-description (format "Here's what I can %s:" verb) fn-map))
@@ -147,23 +148,20 @@
     (when-let [tokens (seq (edn/read-string (str "(" (-> s
                                                          (str/replace "“" "\"") ; replace smart quotes
                                                          (str/replace "”" "\"")) ")")))]
-      (println (u/format-color 'magenta tokens))
       (apply apply-metabot-fn tokens))))
 
 
 ;;; # ------------------------------------------------------------ Metabot Input Handling ------------------------------------------------------------
 
 (defn- message->command-str [{:keys [text]}]
-  (u/prog1 (when (seq text)
-             (second (re-matches #"^mea?ta?boa?t\s+(.*)$" text)))
-    (println (u/format-color 'yellow <>))))
+  (when (seq text)
+    (second (re-matches #"^mea?ta?boa?t\s+(.*)$" text))))
 
 (defn- respond-to-message! [message response]
   (when response
     (let [response (if (coll? response) (str "```\n" (u/pprint-to-str response) "```")
                        (str response))]
       (when (seq response)
-        (println (u/format-color 'green response))
         (slack/post-chat-message! (:channel message) response)))))
 
 (defn- handle-slack-message [message]
@@ -181,14 +179,13 @@
 
 (defn- handle-slack-event [socket start-time event]
   (when-not (= socket @websocket)
-    (println "Go home websocket, you're drunk.")
+    (log/debug "Go home websocket, you're drunk.")
     (s/close! socket)
     (throw (Exception.)))
 
   (when-let [event (json/parse-string event keyword)]
     (when (and (human-message? event)
                (> (event-timestamp-ms event) start-time))
-      (println (u/pprint-to-str 'cyan event))
       (binding [*channel-id* (:channel event)]
         (do-async (handle-slack-message event))))))
 
@@ -197,10 +194,8 @@
 
 (defn- connect-websocket! []
   (when-let [websocket-url (slack/websocket-url)]
-    (log/info "Launching MetaBot... 🤖")
     (let [socket @(aleph/websocket-client websocket-url)]
       (reset! websocket socket)
-      (log/info "Connected to WebSocket.")
       (d/catch (s/consume (partial handle-slack-event socket (System/currentTimeMillis))
                           socket)
           (fn [error]
@@ -221,7 +216,6 @@
 (defn- start-websocket-monitor! []
   (future
     (reset! websocket-monitor-thread-id (.getId (Thread/currentThread)))
-    (log/debug "Monitor thread ID ->" (.getId (Thread/currentThread)))
     ;; Every 2 seconds check to see if websocket connection is [still] open, [re-]open it if not
     (loop []
       (Thread/sleep 500)
@@ -229,7 +223,7 @@
         (try
           (when (or (not  @websocket)
                     (s/closed? @websocket))
-            (log/info "MetaBot WebSocket is closed. < Thread" (.getId (Thread/currentThread)) ">")
+            (log/debug "MetaBot WebSocket is closed.  Reconnecting now.")
             (connect-websocket!))
           (catch Throwable e
             (log/error "Error connecting websocket:" (.getMessage e))))
@@ -240,7 +234,8 @@
 
    This will spin up a background thread that opens and maintains a Slack WebSocket connection."
   []
-  (when (slack/slack-token)
+  (when (and (setting/get :slack-token)
+             (= "true" (setting/get :metabot-enabled)))
     (log/info "Starting MetaBot WebSocket monitor thread...")
     (start-websocket-monitor!)))
 

src/metabase/models/setting.clj

@@ -5,6 +5,7 @@
             [korma.core :as k]
             [metabase.config :as config]
             [metabase.db :refer [exists? sel del]]
+            [metabase.events :as events]
             [metabase.models [common :as common]
                              [interface :as i]]
             [metabase.setup :as setup]
@@ -128,7 +129,7 @@
     (if-let [v (clojure.core/get settings k)]
       (set k v)
       (delete k)))
-  settings)
+  (events/publish-event :settings-update settings))
 
 (defn set*
   "Set the value of a `Setting`, deleting it if VALUE is `nil` or an empty string."

zen.md

+# The Zen of Metabase
+
+- Give the user value as soon as possible
+
+- Structure things so that we can automagically infer things for the user
+
+- Don’t ask the user for information the system should already know
+
+- Make it easy for the user to do the right thing
+
+- Don’t leave the user booby traps
+
+- Go the extra mile to make the user experience pleasant 
\ No newline at end of file