Merge pull request #780 from metabase/update_docs

Add documentation

README.md

@@ -12,7 +12,7 @@ The Report Server does not deal with getting data into a database or data wareho
 
 The report server does not collect web page views or mobile events, though it can help you understand conversion funnels, cohort retention and use behavior in general once you have collected these events into a database. 
 
-See the [Data Warehouse Guide](docs/DATAWAREHOUSING.md) for more information and advice.
+See the [Data Warehouse Guide](docs/data-warehousing.md) for more information and advice.
 
 # Security Disclosure
 
@@ -46,15 +46,15 @@ On logging in, you will be asked a set of questions that will set up a user acco
 
 Once you have added this connection, you will be taken into the app and you'll be ready to ask your first question. 
 
-For more information or troubleshooting, check out the [Installation Guide](docs/INSTALLATION.md)
+For more information or troubleshooting, check out the [Installation Guide](docs/installation-guide.md)
 
 # Getting Started
 
-Follow our [Getting Started](docs/GETTINGSTARTED.md) guide to learn how to use the Report Server.
+Follow our [Getting Started](docs/getting-started.md) guide to learn how to use the Report Server.
 
 # Contributing
 
-To get started with a development installation of the Query Server and learn more about contributing, please follow the instructions at our [Developers Guide](docs/DEVELOPERS.md). 
+To get started with a development installation of the Query Server and learn more about contributing, please follow the instructions at our [Developers Guide](docs/developers-guide.md). 
 
 # Extending and Deep Integrations
 

docs/GETTINGSTARTED.md

-# Before you start
-
-This guide assumes you have a database you have access to and it is set up correctly. If not, please follow the instructions in the [Installation Guide](INSTALLATION.md)
-
-# Understanding what data you have
-
-Initially, let's see what data you have available. The Explore section of the app allows you to see which tables you have available, look at all the rows in a given table, and drill down to individual rows. 
-
-* Click `explore`
-* Note that all of your tables are there
-* Click on one
-* Note the pagination
-* Try getting the next page
-* Note that you can filter these pages
-* Try to filter by a column
-* if it’s a date
-* if it’s a category
-* Note that any IDs or Foreign keys are clickable
-* Click on one
-* Note that all fields are present
-* We can click on any FKs
-* Any urls are clickable
-* Note the `Linked Entites` on the bottom
-* Click on one of these and note that below are a bunch of that entities linked objects
-
-# Asking a Question
-
-When you have a specific question you are trying to answer, you can use the Card section of the application. Here you can ask a specific question of a given table of data you have. We'll start with the simplest possible question you can ask, "How many X are there?".
-
-* Click `Cards`
-* Click `Create New`
-* Select a database
-* if you only have a single database, this step happens automatically
-* Select a table
-* See the bare rows
-* click run
-* note that this allows you to see all of the rows in a table
-* Select `total count`
-
-# Saving a Question to a Dashboard
-
-Assuming this is something you'll want to keep tabs on regularly, or share regularly, you can add it to a dashboard. Dashboards are collections of questions you have saved that you expect to look at as a group or that everyone in your organization can look at. 
-
-* Save
-* Add it to a dashboard
-* Give it a name
-
-* Go to your newly created dashboard
-* click `Dashboards`
-* click your new dashboard
-* Note that your card is there
\ No newline at end of file

docs/README.md

+## For Users
+
+* [Getting Started](getting-started.md)
+* [Users Guide](users-guide.md)
+
+## For Administrators
+
+* [Installation Guide](installation-guide.md)
+* [Administration Guide](administration-guide.md)
+* [Metadata Guide](metadata-guide.md)
+
+## Advanced Topics 
+
+* [Data Warehousing Guide](data-warehousing.md)
+* [ETL Guide](etl-guide.md)
+
+## For Developers
+* [Developers Guide](developers-guide.md)
+* [API Reference](uberdoc.html)
+
+## Reference
+* [Anonymous Information Collection Reference](information-collection.md)
\ No newline at end of file

docs/administration-guide.md

+#Administration Guide
+---
+If you haven't already installed a copy of Metabase, you'll need to do so.  If you need help doing that, check out our [Installation Guide](installation-guide.md).  
+
+##Managing Databases
+---
+
+To see a list of all databases click on "Databases".
+
+##Adding a Database Connection
+
+Click on "Add database".  You can currently add the following types of databases:
+
+* Amazon RDS
+* H2
+* MONGODB
+* MySQL
+* Postgres
+* MongoDB
+
+For each type of database, you'll need the appropriate connection information.  For help finding your database type's connection information, check out our [Installation Guide](installation-guide.md).
+
+##SSL
+
+Metabase automatically tries to connect to databases with and without SSL.  If it is possible to connect to your database with a SSL connection, Metabase will mark an SSL connection as the default for your database. If you'd like, you can change this setting later.  
+
+###Database Analysis
+
+When connecting with your database, we try to decipher the field types in your table based on their name.  We also take a sample of each table to look for URL's, json, encoded strings, etc.  If we classify a field wrong, you can always edit it later.  
+
+###Metadata Syncing
+
+Metabase automatically syncs the table and column information from your database every night.  
+
+To sync manually:
+
+1. Click on your database.
+2. Click on "Connection Details"
+3. Select "Sync"
+
+###Deleting Databases
+
+You can delete a database from Metabase by either clicking "Remove this Database" under "Connection Details".  You can also delete it by hovering over the database row in the list and clicking on the red "Delete" button.
+
+**Caution: Deleting a database is irreversible!  All saved questions and dashboard cards based on the database will be deleted too.**
+
+##Setting up Email
+
+---
+
+Once you connect your database to Metabase, you'll want to configure your email settings.  Metabase uses email to reset passwords, onboard new users, and notify you when something happens.  
+
+###Configuring your Email Account
+
+* Set up an smtp server
+* If you use Google Apps, you can find your credentials...
+* If you use SES, your credentials are located
+* If you use Mandrill, your credentials are...
+* SSL is preferred because it is more secure and gives your account exta security and protection from threats.
+* If your email service has a whitelist of email addresses that are allowed to send email, be sure to add the "Sender of System Notifications" setting to a whitelisted email address to ensure you receive all messages from Metabase.  
+
+##Metadata Editing
+
+---
+*For an in-depth description of Metabase's understanding of metadata and how it uses it, check out our [Metadata Guide](metadata-guide.md)*
+
+###What is metadata?
+**Metadata** is data about other data.  It's data that tells you about the data found in your database.
+###Tables 
+
+* Add descriptions to tables to let people know type of data a table contains and how it can be used. 
+* Descriptions are displayed in the data model reference.
+* You can control visibility of metadata by hiding it. 
+* If you hide a field, give Metabase a reason so it understands why a field is not being included.
+
+###Fields 
+
+* If Metabase misclassified the type of a field, you can update it here. 
+* Add a description to a field, so users know what data it contains.
+* Descriptions are extra helpful when fields have values that are abbreviated or coded in a particular format.
+* Descriptions are displayed in the data model reference.
+
+##Managing User Accounts
+---
+
+Click **People** from [Where do they click from] to see a list of all user accounts in your organization.
+
+* To add a new user account, click **Add Person** and enter their name and email address.  
+
+* New users will receive an email welcoming them to Metabase and a link to configure their password.
+* To delete a user's account, click **Remove**.  Deleting an account will mark it as inactive and prevent it from being used in the future - but it won't delete the user's cards or dashboards.
+
+* To make an existing user an administrator, click **Grant Admin**
+* To remove administrator privileges from a user, select **Revoke Admin**
+
+##Backing up Metabase Application Data 
+---
+###If you're using an Embedded Database
+Find the file "metabase.db.h2.db".  If your system is inactive, you can make a copy directly.  If your system is active, shut down the Metabase process and make a backup copy of the file.  Then, restart the server.
+
+###If you're using Amazon RDS for the Database Application
+Turn on automated RDS backups.  
+
+###If you're using a self-managed PostgreSQL or MySQL database
+Back up your database as you would to any other PostgreSQL or MySQL database. 
+
+##Settings
+---
+###Base URL
+The **base URL** is used in emails to allow users to click to their specific instance.  Include the protocol (http vs https) to make sure it is reachable. 
+
+###Connection Timezone
+The **connection timezone** sets the default time zone for displaying times.  The timezone is used when doing date breakouts.  
+
+Setting the default timezone will not change the timezone of any data in your database.  If the underlying times in your database aren't assigned to a timezone, Metabase will use the connection timezone as the default timezone.  
+
+### Name used for the instance
+If you want to name an instance, you can do so under settings.  Many teams use the name of their company, but the choice is yours!
+
+
+## Common Problems
+
+### Startup fails due to Migrations being locked
+
+Sometimes Metabase will fail to startup due to a lingering lock. 
+
+Solution:
+Run  `java -cp metabase-master-2015-08-06-3da1178.jar org.h2.tools.Server -webPort 3000` in the commandline
+
+open the web console (it prints a url)
+
+connect to JDBC URL jdbc:h2:PATH/TO/metabase.db;IFEXISTS=TRUE  # note the .h2.db suffix is omitted
+blank username, blank password
+
+run `delete from databasechangeloglock;`
+
+kill the h2 server and restart metabase.

docs/etl-guide.md

+## ETL Guide
+* Normalized vs denormalized data
+* Database Views for fun and profit
+	* ETL jobs vs database views
+		* write load vs read load
+		* performance implications
+		* when to use views
+	* how to create them
+		* MySQL
+		* PostgreSQL
+	* Common examples
+		* tables with an is_deleted flag
+		* user table with a roles field
+		* test accounts + data
+* Dealing with events
+	* pain on read vs pain on write
+	* deciding what to collect
+	* suggested event format
+	* enriching events
+	* preparing for cohort analysis

docs/getting-started.md

+#Welcome to Metabase
+Metabase lets you ask questions about your data, drill into specific records, and dashboards to help keep you organized.  
+
+##Step 1: Installing Metabase
+Metabase uses Java to operate, so you'll need at least version 1.6 or later.  Not sur what version of Java you're using?  No problem.  Here's how to check: 
+
+###Mac Users: 
+In Terminal, insert the command prompt: "java -version".  You will receive a message similar to: 
+
+    java version "1.60_65"
+    Java (TM) SE Runtime Environment (build 1.6.0_65-b14-466.1-11M4716)
+    Java HotSpot (TM) 64-Bit Server VM (build 20.65-b04-466.1, mixed mode)
+    
+As long as the version is at least 1.6, you're all set to go!  
+
+###Windows Users:
+Under Programs, click on the "Java" icon.  Click on "About" and then find the version number listed.  If you're using version 1.6 or greater, then you're good to go! 
+
+If you don't have the latest version of Java, download it at: [https://java.com/en/download/](https://java.com/en/download/)
+
+Once you take care of checking the version of java on your computer, **Download the Metabase file from [www.metabase.com/download](www.metabase.com/download).**  Place the Metabase JAR in the directory.  Run the command `java -jar metabase.jar` to create a file called "metabase.db.h2.db".  **This file contains important application data, so don't delete it!**
+
+Now that you have Metabase installed, you can sync it to your database.  Go to [http://localhost:3000](http://localhost:3000) to connect your database.  
+
+##Step 2: Configuring your Instance
+Create a Metabase account by entering your name, email, and super-secretive password—we don't recommend using password123!
+
+Now Metabase needs to find your data.  Where does your data live?  Depending on where you keep your database, the steps to connect with Metabase vary.  Not to worry, though, we outlined the configuration steps for each of the major platforms.  
+
+###If you use Heroku: 
+
+1. Go to [https://postgres.heroku.com/databases](https://postgres.heroku.com/databases).  
+2. Click on the database you want to connect to Metabase. 
+3. Write down the following information based on your database:
+    * Hostname
+    * Port
+    * Username
+    * Database Name
+    * Password
+
+You'll need to input this information into the remainder of the Metabase form.  
+
+###If you use AMAZON RDS:
+
+1. Go to your AWS Management Console. 
+    * Need help finding that?  Visit [https://**My_AWS_Account_ID**.signin.aws.amazon.com/console](https://**My_AWS_Account_ID**.signin.aws.amazon.com/console).  Be sure to insert your own AWS Account ID, though! 
+2.  Under "Database" services, click "RDS". 
+3.  Then click "Instances".
+4.  Select the database you want to connect to Metabase.  
+5.  Write down the following information based on your database:
+    * Hostname - This is listed as the "Endpoint" parameter
+    * Port - Find the port parameter under "Security and Network"
+    * Username - Find this under "Configuration Details"
+    * Database Name - Find this under "Configuration Details"
+    * Password - Ask your database administrator for the password. 
+
+You'll need this information to finish syncing Metabase with your database.  
+
+###If you use another REMOTE MySQL or POSTGRES: 
+
+1. Ask your database administrator (or check your own records) for the following information
+    * Hostname
+    * Port
+    * Username
+    * Database Name
+    * Password
+
+You'll need to input this information into the Metabase form.  
+
+###If you use MONGODB:
+
+1.  Collect the following information about the database you'd like to connect to Metabase. 
+    * Hostname
+    * Port
+    * Username
+    * Database Name
+    * Password
+
+Metabase needs this information to finish connecting to your database.  
+
+###If you use H2:
+
+1.  You'll need the file path for your database. 
+
+**Now that you have your database information,** use the information to fill out the remainder of the form.  
+
+After you enter your database's information, Metabase will try to connect to your database and validate the credentials.  If you get a validation error, no need to panic.  Validation errors occur when the wrong credentials are entered.  Simply double check the spelling and punctuation of the information you entered and try to connect to the database once more. 
+
+Once Metabase successfully connects to your database, it'll run a few queries against your database to build a model of your data.  Click the prompt "continue" to see what data Metabase found in your database!
+
+##Step 3: Discovering your Data
+
+If you look at your homepage, you'll see the different tables that Metabase was able to find in your data source set and pulled for you.  Click a table you want to learn more about.  Note the number of rows.
+
+*Figure 1*
+
+![Figure 1: Metabase Home with Tables](images/MetabaseTable1.png =500x300)
+
+For example, in the sample data set above, the database (data source) is "Bird Impact" (it's a database of the number of collisions between birds and aircrafts).  The table is "strikes" and the number of rows is 151,069. 
+
+##Step 4: Asking your Question 
+
+By clicking on an individual table, you enter the interface (pictured below) that allows you to ask Metabase questions based on your data.  
+
+*Figure 2*
+
+![Figure 2: Metabase Initial Table View](images/MetabaseTable2.png =500x300)
+
+Your screen will reflect the data source you connected to Metabase and the table you selected.  In our example, the data source is "Bird Impact" and the table is "strikes" (exactly what we selected on the previous screen). 
+
+For now, let's start with a basic question using our data set.  How many collisions were there between aircrafts and birds?  More precisely, this question translates to "How many records are in the table 'strikes'?"  To find the number, we want to _filter_ the data by **the field we are interested in** (which is "Birds Struck" for our example).  Our operator is **"is"** because we want to know the value (not whether it is greater or less than a number, etc.).  The value is left blank because we don't know the value.  We want to view the **Count** because we want to know the number (not a listing of every incident). 
+
+*Figure 3*
+
+![Metabase Question Format](images/MetabaseQuestion1.png)
+
+After you select the different components of your question, click **Run query**.
+
+*Figure 4*
+
+![Answer to Query](images/MetabaseQuestion1Answer.png =500x300)
+
+There were 151,069 collisions between aircraft and birds.  Stated in database parlance, there are 151,069 records in the table.  Does this number look familiar?  When selecting a table on your homepage in Step 3 (See *Figure 1*) 151069 was labeled as "total" and listed next to the table "strikes".  
+
+**The total number listed next to each table is the number of records.  Each record is an iteration of the event your database records.**
+
+Metabase can present the answers to your questions in a variety of formats.  To change the format, select one of the options from the dropdown menu in the top left hand corner of the screen next to **Visualization**.  
+
+*Figure 5*
+
+![Visualization Dropdown Menu](images/VisualizationMenu.png)
+
+
+Not every format is the best way to show an answer to a question.  If Metabase think that's the case with a specific question and display format, the format choice will appear faded.  For example, it wouldn't make sense to show the number of collisions between aircraft and birds as a singular bar graph.
+
+You can "group" your data into categories.  Click **Add a grouping** to select the category to filter your answer by.  Metabase will analyze your database to discover valid categories for adding filters to your questions.  For example, we can ask Metabase to tell us the number of collisions between aircraft and birds grouped by the weather conditions.  The table "strikes" already had a field to record weather conditions labeled "PRECIP" (short for "precipitation"). 
+
+*Figure 6*
+
+![Group by Feature](images/GroupingFeature.png)
+
+**When you make any changes to the question (called the "query"), the blue "Run query" button reappears.** Click it to refresh the answer and to find the answer to your new query. 
+
+
+
+ 
+   
+
+
+