This project is mirrored from https://github.com/metabase/metabase.
Pull mirroring updated .
- Mar 12, 2021
-
-
Robert Roland authored
It's possible for the scheduler to get in a weird state if the sync fails while it executes. This change makes it *only* recreate a job/task if the schedule has changed or if it is missing. Previously, clearing the state at every start had bad effects if the JVM had terminated during the sync. Adds a vector of Exception classes that signal a "fatal" exception during sync for a specific database. If these exceptions occur, the sync for that database stops and will pick up next time. This will have to be expanded per driver, but I don't see a way around that, as each driver will have its own, unique way of failing. metabase/metabase#14817
-
- Oct 23, 2020
-
-
Cam Saul authored
-
- Apr 02, 2019
-
-
Cam Saul authored
-
- Feb 04, 2019
-
-
Cam Saul authored
-
- Jul 09, 2018
-
-
Cam Saul authored
-
- Jan 02, 2018
-
-
Ryan Senior authored
Previously we configured each database to have separate trigger + job instances for sync+analyze and field values scanning. We were using the default threadcount and "do nothing" upon misfires. This resulted in a race condition like below. By default, sync and field-values scans are scheduled at 50 minutes past the hour (two separate jobs). As an example assume we have 10 databases configured. This would result in 20 jobs firing at 50 minutes past the hour. The order of execution of these jobs is not defined. By default, we have 4 threads to execute these jobs. If the 4 threads are busy when a trigger occurs, it will wait up to one minute (the default). If it can't get a thread in 1 minute, it will misfire. The policy we had setup will discard the job when it misfires. The end result of this is a race condition where we could potentially not run sync on many databases and also be running 4 scans or syncs at the same time, potentially causing memory issues. This commit switches to a single job for sync and a single job for field value scans. Every database will have a trigger for sync and a trigger for field values, but only 1 instance of sync and field values scanning will run at a time. In the event of a misfire, it will execute it as soon as it's able and only once.
-
- Jun 15, 2016
-
-
Tom Robinson authored
-