<html>
<head>
<title>firstworks   SQL Relay - Query Routing</title>
<link href="../css/styles.css" rel="stylesheet">
</head>
<body>

<br>
<span class="heading1">Query Routing</span><br>

<p>SQL Relay's query routing feature allows you to send one set of queries to one database, another set of queries to another.</p>

<p>How is this useful?  Here are some examples of what you can do:</p>

<ul>
<li>Send inserts, updates and deletes to a master database and distribute selects over a set of slaves.</li>
<li>Provide a single point of access to multiple databases, even different types of databases.</li>
</ul>

<p>These are not the only things you can do.  Use your imagination.</p>

<br>
<span class="heading1">How it Works</span><br>

<p>Normally, SQL Relay maintains a set of persistent connections to a database or database cluster and distributes queries over the connections.  The databases to connect to are defined in a configuration file.</p>

<p>When used in query routing mode, SQL Relay is configured to distribute queries over other instances of SQL Relay running in standard mode.</p>

<p>For instance, you can set up one instance of SQL Relay to maintain connections to a master database and another instance of SQL Relay to maintain connections to a pool of slaves, then set up a third instance of SQL Relay to route queries to the other 2 instances.</p>

<blockquote><img src="../images/router.png"></blockquote>

<br>
<span class="heading1">General Configuration</span><br>

<p>Each instance of SQL Relay that will be used as a query router must omit the <b>connection</b> tag and include a single <b>router</b> tag.  The <b>router</b> tag must contain a set of <b>route</b> tags.  Each <b>route</b> tag defines an instance of SQL Relay to route queries to and contains a set of <b>query</b> tags defining which queries to route to it.</p>

<p>When deciding where to route a query, SQL Relay runs through the <b>query</b> tags, in the order that they appear in the configuration file, and compares the query to the regular expression defined in the tag.  If the query matches a regular expression defined in a <b>query</b> tag inside of a <b>route</b> tag, it will be routed to the instance of SQL Relay defined in the <b>route</b> tag.</p>

<p>See the examples below.  Note that the regular expressions in the
examples below aren't perfect.  They are not case insensitive, for example, nor
do they necessarily cover all possible cases.  In some cases, better, or more
regular expressions could be written.  In other cases, developers may need to
structure their queries so that they are guaranteed to match.</p>

<span class="heading1">Sending Inserts, Updates and Deletes to a Master
Database and Distributing Selects Over a Set of Slaves</span>

<p>Here's an example configuration for sending inserts, updates and delete's to
a master database and distributing selects over a set of slaves.</p>

<p>In this example, 3 instances of SQL Relay are defined: one to maintain
connections to the "master" database, another to maintain connections to a
set of "slave" databases, and another to route queries over the "master" and
"slave" instances.  Clients would connect to the "router" instance (on port 9001
or to the /tmp/router.socket unix socket).  In this example, all 3 instances
of SQL Relay run on the same machine, but in reality, all 3 could run on
separate machines.</p>

<blockquote>
@sqlrelay-router.conf.html@
</blockquote>

<span class="heading1">Provide a Single Point of Access to Multiple Databases</span>

<p>Here's an example configuration providing a single point of access to
multiple databases.</p>

<p>In this example, we provide a single point of access to the MySQL database
"mysqldb" and the PostgreSQL database "postgresqldb".  Since both databases
support the "database.tablename" syntax, we can use that format when specifying
tables and send all queries containing "mysqldb." to the MySQL database and
all queries containing "postgresqldb." to the PostgreSQL database.</p>

<blockquote>
@sqlrelay-router-multipledb.conf.html@
</blockquote>

<span class="heading1">Quirks and Limitations</span>

<br><br>
<span class="heading2">Query Normalization</span>

<p>To make pattern matching easier, SQL Relay "normalizes" the query before
matching it against the pattern.  The original query is run against the database
but when matched against the pattern, whitespace is compresssed and the entire
query is converted to lower-case.</p>

<p>When matching query operators, you must use lower-cased versions of them such
as "select", "insert", "and", "or", etc.  When matching table names, you must
use a lower-cased version of the table-name.</p>

<br><br>
<span class="heading2">Examples are Limited</span>

<p>The examples above are good introductions, but don't cover every scenario.
For example, they expect the query to begin at the beginning of a line with
no preceeding comments or spaces.  You may have to write more complex patterns
for your applications.</p>

<br><br>
<span class="heading2">Perl Compatible Regular Expressions</span>

<p>SQL Relay is built upon the Rudiments library.  Rudiments can be built with
or without support for libpcre which provides support for Perl Compatible
Regular Expressions.  PCRE's are more powerful than standard posix regular
expressions and have many more operators.</p>

<p>As such, if you copy an sqlrelay.conf file from a machine where Rudiments was
compiled with PCRE support to a machine where Rudiments wasn't compiled with
PCRE support, then it's possible that your patterns may not work on the new
machine.</p>

<p>To make matters worse, sufficiently old versions of the posix regular
expression functions had fewer operators than modern versions.  So, even if
Rudiments isn't using PCRE's, it's not impossible that after copying a
sqlrelay.conf file from a fairly modern OS to an antique, the patterns won't
work on the antique machine either.</p>

<p>The examples above ought to work with PCRE's and all versions of posix
regular expressions.</p>

<br><br>
<span class="heading2">Selects Not Showing Changes</span>

<p>In the scenario above where DML/DDL is sent to the master database and
selects are distributed over slaves, an unintuitive thing can happen.</p>

<p>If you begin a transaction and do several inserts, updates and deletes,
you'll find that if you do a select, you will not see your changes.  This is
because in a master-slave configuration, changes to the database are not
pushed out to the slaves until the changes have been committed.  Since your
selects are being run against the slaves, you must first commit before your
changes will be visible.</p>

<br><br>
<span class="heading2">Commits and Rollbacks</span>

<p>Behind the scenes, the router maintains parallel transactions on each of the
databases that it is routing queries to.  When the client issues a begin, commit
or rollback, the router issues a begin, commit or rollback to each of the
databases.  Similarly, if the client turns auto-commit on or off, the router
turns auto-commit on or off on each of the databases.</p>

<p>There are scenarios where a commit, rollback or auto-commit on/off command
could succeed on some of the databases and fail on others.  Some databases
have a 2-phase commit feature to handle these scenarios.  With 2-phase commit,
you can roll back a commit until you do second commit.  Many databases don't
support 2-phase commit though.  At present, SQL Relay doesn't currently support
2-phase commit for any databases.  So, currently, SQL Relay doesn't handle these
scenarios very well.  It returns false, indicating than the operation failed,
but there's no good way to know which databases succeeded and which failed.
Ideally, SQL Relay would remove the database that failed, stop sending queries
to it, notify someone that it has gotten out of sync and log all of the queries
that would have gone to it so they can be replayed later, or something like
that.  But it doesn't currently do any of that.</p>

<p>Since queries may be routed to different kinds of databases, the router has
to employ some tricks to maintain parallel transactions.  Some databases run
in auto-commit mode by default and must be issued a "begin" query to start a
transaction.  Other databases implicitly start a new transaction when a client
logs in and after each commit or rollback.  If any of the databases being
routed to require a "begin" query to start a transaction, then the ones that
don't are put in auto-commit mode when the client logs in and after each commit
or rollback and are taken out of auto-commit mode when the client sends a begin
query.  If none of the databases being routed to require a "begin" query to
start a transaction, then the databases are not put in auto-commit mode when
the client logs in or after each commit or rollback.  Rather, transactions are
implicitly started by the database.  For example, if your client application
is using a router which routes queries over both PostgreSQL and Oracle databases,
then since PostgreSQL requires "begin" queries, you must use a "begin" query to
start a transaction, even if your app only intends to send queries which would
be run against Oracle.  Conversely, if your client application is using a router
which only routes queries over a set of Oracle databases, then you do not have
to use "begin" queries.</p>

<br><br>
<span class="heading2">Stored Procedures</span>

<p>It's possible to use stored procedures with SQL Relay's query routing
feature.  However, since stored procedures are run on the database, SQL Relay
can't route the individual queries run inside the stored procedure.  So, the
stored procedure and all queries run inside of it will be run against whichever
database it was routed to.</p>

</body>
</html>
