Merge pull request #5718 from metabase/add_troubleshooting_guide_improvements

add some useful snippets to the troubleshooting guides

docs/operations-guide/start.md

@@ -3,7 +3,6 @@
 
 *  [How to install Metabase](#installing-and-running-metabase)
 *  [How to upgrade Metabase](#upgrading-metabase)
-*  [Tips for troubleshooting various issues](#troubleshooting-common-problems)
 *  [Configuring the application database](#configuring-the-metabase-application-database)
 *  [Migrating from using the H2 database to MySQL or Postgres](#migrating-from-using-the-h2-database-to-mysql-or-postgres)
 *  [Running database migrations manually](#running-metabase-database-migrations-manually)
@@ -68,74 +67,6 @@ Step-by-step instructions on how to upgrade Metabase running on Elastic Beanstal
 #### [Upgrading Heroku deployments](running-metabase-on-heroku.md#deploying-new-versions-of-metabase)
 Step-by-step instructions on how to upgrade Metabase running on Heroku.
 
-# Troubleshooting Common Problems
-
-### Metabase fails to start due to database locks
-
-Sometimes Metabase will fail to complete its startup due to a database lock that was not cleared properly. The error message will look something like:
-
-    liquibase.exception.DatabaseException: liquibase.exception.LockException: Could not acquire change log lock.
-
-When this happens, go to a terminal where Metabase is installed and run:
-
-    java -jar metabase.jar migrate release-locks
-
-in the command line to manually clear the locks. Then restart your Metabase instance.
-
-### Metabase fails to start due to PermGen OutOfMemoryErrors
-
-On Java 7, Metabase may fail to launch with a message like
-
-    java.lang.OutOfMemoryError: PermGen space
-
-or one like
-
-    Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler
-
-If this happens, setting a few JVM options should fix your issue:
-
-    java -XX:+CMSClassUnloadingEnabled -XX:+UseConcMarkSweepGC -XX:MaxPermSize=256m -jar target/uberjar/metabase.jar
-
-You can also pass JVM arguments by setting the environment variable `JAVA_TOOL_OPTIONS`, e.g.
-
-    JAVA_TOOL_OPTIONS='-XX:+CMSClassUnloadingEnabled -XX:+UseConcMarkSweepGC -XX:MaxPermSize=256m'
-
-Alternatively, you can upgrade to Java 8 instead, which will fix the issue as well.
-
-### Metabase fails to start due to Heap Space OutOfMemoryErrors
-
-Normally, the JVM can figure out how much RAM is available on the system and automatically set a sensible upper bound for heap memory usage. On certain shared hosting
-environments, however, this doesn't always work perfectly. If Metabase fails to start with an error message like
-
-    java.lang.OutOfMemoryError: Java heap space
-
-You'll just need to set a JVM option to let it know explicitly how much memory it should use for the heap space:
-
-    java -Xmx2g -jar metabase.jar
-
-Adjust this number as appropriate for your shared hosting instance. Make sure to set the number lower than the total amount of RAM available on your instance, because Metabase isn't the only process that'll be running. Generally, leaving 1-2 GB of RAM for these other processes should be enough; for example, you might set `-Xmx` to `1g` for an instance with 2 GB of RAM, `2g` for one with 4 GB of RAM, `6g` for an instance with 8 GB of RAM, and so forth. You may need to experiment with these settings a bit to find the right number.
-
-As above, you can use the environment variable `JAVA_TOOL_OPTIONS` to set JVM args instead of passing them directly to `java`. This is useful when running the Docker image,
-for example.
-
-    docker run -d -p 3000:3000 -e "JAVA_TOOL_OPTIONS=-Xmx2g" metabase/metabase
-
-### Metabase fails to connect to H2 Database on Windows 10
-
-In some situations the Metabase JAR needs to be unblocked so it has permissions to create local files for the application database.
-
-On Windows 10, if you see an error message like
-
-    Exception in thread "main" java.lang.AssertionError: Assert failed: Unable to connect to Metabase DB.
-
-when running the JAR, you can unblock the file by right-clicking, clicking "Properties," and then clicking "Unblock."
-See Microsoft's documentation [here](https://blogs.msdn.microsoft.com/delay/p/unblockingdownloadedfile/) for more details on unblocking downloaded files.
-
-There are a few other reasons why Metabase might not be able to connect to your H2 DB. Metabase connects to the DB over a TCP port, and it's possible
-that something in your `ipconfig` configuration is blocking the H2 port. See the discussion [here](https://github.com/metabase/metabase/issues/1871) for
-details on how to resolve this issue.
-
-
 # Configuring the Metabase Application Database
 
 The application database is where Metabase stores information about users, saved questions, dashboards, and any other data needed to run the application.  The default settings use an embedded H2 database, but this is configurable.

docs/troubleshooting-guide/application-database.md

+## Specific Problems:
+
+
+### Metabase fails to start due to database locks
+
+Sometimes Metabase will fail to complete its startup due to a database lock that was not cleared properly. The error message will look something like:
+
+    liquibase.exception.DatabaseException: liquibase.exception.LockException: Could not acquire change log lock.
+
+When this happens, go to a terminal where Metabase is installed and run:
+
+    java -jar metabase.jar migrate release-locks
+
+in the command line to manually clear the locks. Then restart your Metabase instance.
+
+### Metabase H2 application database gets corrupted
+
+Because H2 is an on-disk database, it is sensitive to filesystem errors. Sometimes drives get corrupted, or the file doesn't get flushed correctly, which can result in a corrupted database. In these situations, you'll see errors on startup. These vary, but one example is 
+```
+myUser@myIp:~$ java -cp metabase.jar org.h2.tools.RunScript -script whatever.sql -url jdbc:h2:~/metabase.db
+Exception in thread "main" org.h2.jdbc.JdbcSQLException: Row not found when trying to delete from index """"".I37: ( /* key:7864 */ X'5256470012572027c82fc5d2bfb855264ab45f8fec4cf48b0620ccad281d2fe4', 165)" [90112-194]
+    at org.h2.message.DbException.getJdbcSQLException(DbException.java:345)
+    [etc]
+```
+
+Not all H2 errors are recoverable (which is why if you're using H2, _please_ have a backup strategy for the application database file). To attempt to recover a corrupted H2 file, try the below.
+
+```
+java -cp metabase.jar org.h2.tools.Recover
+mv metabase.db.mv.db metabase.old.db
+touch metabase.db.mv.db
+java -cp target/uberjar/metabase.jar org.h2.tools.RunScript -script metabase.db.h2.sql -url jdbc:h2:`pwd`/metabase.db
+```
+
+NOTE: If you are using a legacy Metabase H2 application database (where the database file is named 'metabase.db.h2.db'), use the below instead. 
+
+```
+java -cp metabase.jar org.h2.tools.Recover
+mv metabase.db.h2.db metabase.old.db
+touch metabase.db.h2.db
+java -cp target/uberjar/metabase.jar org.h2.tools.RunScript -script metabase.db.h2.sql -url jdbc:h2:`pwd`/metabase.db;MV_STORE=FALSE
+```
+
+
+### Metabase fails to connect to H2 Database on Windows 10
+
+In some situations the Metabase JAR needs to be unblocked so it has permissions to create local files for the application database.
+
+On Windows 10, if you see an error message like
+
+    Exception in thread "main" java.lang.AssertionError: Assert failed: Unable to connect to Metabase DB.
+
+when running the JAR, you can unblock the file by right-clicking, clicking "Properties," and then clicking "Unblock."
+See Microsoft's documentation [here](https://blogs.msdn.microsoft.com/delay/p/unblockingdownloadedfile/) for more details on unblocking downloaded files.

docs/troubleshooting-guide/index.md

@@ -2,8 +2,12 @@
 
 ### [Logging in](loggingin.md)
 
+### [Running Metabase](running.md)
+
 ### [Running Metabase on Docker](docker.md)
 
+### [The Metabase Application Database](application-database.md)
+
 ### [Connecting to databases and data warehouses with Metabase](datawarehouse.md)
 
 ### [Incorrect results due to time zones](timezones.md)

docs/troubleshooting-guide/running.md

 
-## Troubleshooting Process
-1. 
-
 ## Specific Problems:
 
 
-### Specific Problem:
-xxx
-#### How to detect this -
-xxx
-#### How to fix this -
-xxx
\ No newline at end of file
+### Metabase fails to start due to PermGen OutOfMemoryErrors
+
+On Java 7, Metabase may fail to launch with a message like
+
+    java.lang.OutOfMemoryError: PermGen space
+
+or one like
+
+    Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler
+
+If this happens, setting a few JVM options should fix your issue:
+
+    java -XX:+CMSClassUnloadingEnabled -XX:+UseConcMarkSweepGC -XX:MaxPermSize=256m -jar target/uberjar/metabase.jar
+
+You can also pass JVM arguments by setting the environment variable `JAVA_TOOL_OPTIONS`, e.g.
+
+    JAVA_TOOL_OPTIONS='-XX:+CMSClassUnloadingEnabled -XX:+UseConcMarkSweepGC -XX:MaxPermSize=256m'
+
+Alternatively, you can upgrade to Java 8 instead, which will fix the issue as well.
+
+
+### Metabase fails to start due to Heap Space OutOfMemoryErrors
+
+Normally, the JVM can figure out how much RAM is available on the system and automatically set a sensible upper bound for heap memory usage. On certain shared hosting
+environments, however, this doesn't always work perfectly. If Metabase fails to start with an error message like
+
+    java.lang.OutOfMemoryError: Java heap space
+
+You'll just need to set a JVM option to let it know explicitly how much memory it should use for the heap space:
+
+    java -Xmx2g -jar metabase.jar
+
+Adjust this number as appropriate for your shared hosting instance. Make sure to set the number lower than the total amount of RAM available on your instance, because Metabase isn't the only process that'll be running. Generally, leaving 1-2 GB of RAM for these other processes should be enough; for example, you might set `-Xmx` to `1g` for an instance with 2 GB of RAM, `2g` for one with 4 GB of RAM, `6g` for an instance with 8 GB of RAM, and so forth. You may need to experiment with these settings a bit to find the right number.
+
+As above, you can use the environment variable `JAVA_TOOL_OPTIONS` to set JVM args instead of passing them directly to `java`. This is useful when running the Docker image,
+for example.
+
+    docker run -d -p 3000:3000 -e "JAVA_TOOL_OPTIONS=-Xmx2g" metabase/metabase