Since we are not starting a new transaction in conn2 during the
swap step, we need to make sure that if our LOCK query is canceled
due to statement_timeout that conn2's transaction is not left in
a useless error state. Use SAVEPOINT and ROLLBACK TO SAVEPOINT to
avoid this problem.
--no-order now is almost useless, but list it next to --order-by.
--jobs only specifies how to do something, not what to do. On the
same basis probably --no-analyze should be pushed further up.
Note: if original namespace is "foo bar", repack_indexdef gives a bad
result. This is weird as apparently skip_ident can deal with spaces in
a quoted identifier. Committing as I'm going home, will deal with that
later.
Bumped version number to enforce extension re-creation as the SQL has
been modified.
Current limitations:
- Check for namespace existence: on error temp objects are left around
- What happens to the indexes?
- Tests needed.
- Should the default be the GUC default_tablespace instead of pg_default?
This is actually an original pg_repack shortcoming, not a regression.
This simplifies some of the error handling blocks, as now
we can unconditionally use this macro without worrying about multiple
PQclear() calls causing a double-free().
Per discussion with Daniele.
* Use poll() if it is available, or select() otherwise, to
efficiently wait on index builds in worker queries to finish.
* fix off-by-one error when initially assigning workers
* move PQsetnonblocking() calls to setup_workers()
Adds a new --jobs command-line argument to specify how many worker
connections you want. These worker connections should stick around
while processing table(s) in a single database. For each table,
parcel out the indexes to be built among these worker conns,
submitting each CREATE INDEX ... request using PQsendQuery() i.e.
in non-blocking fashion.
Most of this is still rather crude, in particular the
while (num_active_workers) ... loop in rebuild_indexes(), but
it seems to be working, so I'm committing here.
Code merged in, with a few more changes, from the multiple_tables branch.
The multiple --table support and SimpleStringList code is largely
borrowed from pg_dump. (pg_reorg Issue #18).
The concurrent DDL guard is implemented using an auxiliary
database connection (pg_reorg Issue #8) which holds an ACCESS SHARE
lock on the target table while pg_repack conducts the rest of its work.
This patch is a port of Daniele's commit 0be414ad10c32d from his own fork,
"error_reporting" branch.
reorg_all_database can return an error message: in case of any error different
from "missing schema" return the error and keep processing the other databases
instead of printing and stopping the program.
The output of the program is now something like:
$ pg_reorg --all
pg_reorg: reorg database "contrib_regression"
pg_reorg: reorg database "template1" ... skipped: pg_reorg is not installed in the database
Previously, an error while processing any single table would cause
pg_reorg to cause exit() and bail out. Quick summary of fixes:
* get rid of pgut_atexit_push() and pgut_atexit_pop() use, since
we are no longer relying on calling exit() to handle mundane errors
* remove lock_conn_pid variable; we can just use buffer instead
* lock_exclusive() and lock_access_share() now return bool instead of
bailing out on any error
* ERROR-level ereport() or elog() calls now return WARNING instead,
to avoid bailing out unnecessarily
* signature of reorg_cleanup() changed; it no longer needs to take a
void pointer
* check return of strdup() for vxid
* Use pgut_rollback() instead of sending ROLLBACK; command directly
There are still one or two FIXMEs left, including fixing table name
escaping, but I'm committing this much.
Per Issue #18. SimpleStringList code borrowed from pg_dump and a
pending patch to add similar functionality to pg_restore,
clusterdb, vacuumdb, and reindexdb.
The error handling in reorg_one_table() could still be much improved,
so that an error processing a single table doesn't cause pg_reorg to
necessarily bail out and skip further tables, but I'll leave that for
another day.
* KILL_COMPETING_LOCKS was using pg_cancel_backend() instead of
pg_terminate_backend()
* create kill_ddl() function for canceling+terminating any pending
unsafe concurrent DDL, i.e. anyone hanging out waiting for
an ACCESS EXCLUSIVE lock on our table.
* create lock_access_share() function for reliably obtaining an
ACCESS SHARE lock on the target table, killing off any queued
ACCESS EXCLUSIVE lockers in the process via kill_ddl()
* Avoid deadlock possible before we run:
CREATE TABLE reorg.table_xxx AS SELECT ... FROM ONLY ...
by using lock_access_share()
* Fix a few calls in lock_exclusive() which were forgetting to
specify the passed-in connection.
These fixes are related to Issue #8. The main thing remaining AFAIK
is to review or fix some of the unlikely-error handling bits;
most of these should be marked with XXX now.
Fix table locking so that race conditions don't exist between lock
release in primary conn, and lock acquisition in conn2. Also, have
conn2 be in charge of performing the table swap step, to avoid a
similar race.
Part of work for Issue #8.
