Archive for October, 2006


Making connections more manageable

by Jeremy Cole on Tuesday, October 31st, 2006 at 18:51:07 in MySQL, MySQL Patches, Proven Scaling

For the past few weeks off and on, as part of Proven Scaling‘s project to improve the MySQL server, I’ve been helping Joel Seligstein to really dig into the MySQL source code and add some features, in preparation for a much bigger feature coming up (more on that at a future date). He has now finished three smaller projects that have been on Proven Scaling’s and my own to-do list for quite some time: SET CONNECTION STATUS status, KILL connection_id WITH QUERY query_id, and SHOW ... FOR CONNECTION connection_id.1

SET CONNECTION STATUS

This patch adds a new SET CONNECTION STATUS status command, which allows each session to set a status which will be shown in a new Status column in SHOW FULL PROCESSLIST.

This allows a administrators to gain a bit more insight into complex multi-tier architectures, by having essentially a comment for each database connection. In SET CONNECTION STATUS, the status argument may be a complex expression, so CONCAT() and other string manipulations may be used. The current connection status may be retrieved with CONNECTION_STATUS(). Some things which I could imagine putting in the connection status are:

  • in pool — See at a glance which connections are idle in the connection pool, and which are checked out.
  • GET /foo.php — Easily see what request each connection is responsible for.
  • apache pid 672 — Allow you to easily correlate activity on a given server with activity on MySQL, without having to use netstat and friends to track things down.

Of course, there are many more creative people than me to figure out things to do with this useful feature!

KILL thread_id WITH QUERY query_id

In order to make this command useful, the query_id first had to be exposed in SHOW FULL PROCESSLIST. When WITH QUERY query_id is specified in a KILL command, KILL checks that the connection is still executing the query_id you’ve specified (and locks to ensure that it does not start a new query) before killing it. This solves a well-known race condition between SHOW PROCESSLIST and KILL, where the connection may have moved on to a potentially dangerous query to kill, such as a non-transactional UPDATE.

SHOW ... FOR CONNECTION connection_id

This patch extends the SHOW VARIABLES and SHOW STATUS commands with a FOR CONNECTION connection_id clause, which allows a user with either the same credentials as the connection, or with the SUPER privilege to view the connection’s status and variables.

All of these features will be great for debugging production systems, where it can be difficult or impossible to get any insight into what is happening at any given moment in time.

Thanks, Joel, for the hard work!

1 All three patches are against 5.0.26.

MySQL Camp Free Ride goes to Adam Ritter

by Jeremy Cole on Saturday, October 21st, 2006 at 10:54:05 in MySQL, MySQL Camp, Proven Scaling

And, the winner is……

Adam Ritter, of Nashville, Tennessee has been selected by Proven Scaling for a free ride to MySQL Camp!

Adam’s enthusiasm and very quick (within the first few hours of my original post), yet courteous and complete entry impressed us. His excitement to learn, and professionalism sealed the deal. Here’s an excerpt from his entry:

My name is Adam Ritter and I’m a recent Computer Science graduate from UT Knoxville. I’m currently looking for a job and I realized that being a DBA is what I want to do. I had started working with MySQL back in a course on Database Design and Management course I took in school. This course started the fire inside of me but unfortunately, it was over all too soon and I was left wanting to learn more. I’ve read through every intermediate MySQL source I could find, but I’m looking for as much advanced and expert level topics as I can find. I’ve worked on projects utilizing MySQL and have grown very fond of it.

If you’re attending MySQL Camp in a few weeks time, be sure to say hello to Adam!

Thanks, everyone! Keep an eye on my blog early next year for a similar offer for MySQL Conference & Expo 2007!

Free Ride Deadline Extended

by Jeremy Cole on Wednesday, October 18th, 2006 at 13:21:52 in MySQL, MySQL Camp, Proven Scaling

I realized earlier that having the MySQL Camp Free Ride offer open for only 24 hours is a bit harsh, and needlessly short. I’ve extended the deadline to midnight, Friday, October 20 to give everyone a chance.

I’ve also clarified that the offer is open to all, not only to students.

Good luck!

Want a Free Ride to MySQL Camp?

by Jeremy Cole on at 04:47:50 in MySQL, MySQL Camp, Proven Scaling

Have you heard of MySQL Camp at Google HQ in Mountain View, California, November 10-12?

Want to go? Can’t go because it’s out in sunny (ha, ha) California, hundreds or thousands of miles/kilometers from where you live?

Maybe you’re a bright student, or maybe you’re currently unemployed, or maybe you’ve got a job, but you just can’t afford to jet out to Mountain View for the weekend. Maybe you contribute code, bug fixes, tools; maybe you run a MySQL Meetup, and just can’t bear the airfare.

Proven Scaling would like to sponsora one such person to attend MySQL Camp! We’ll pay for your airfare and hotel—meals and transportation are on you. If you think you are qualified, you live somewhere in the USA or Canadab and you know you can make the trip, send us (with “MySQL Camp Free Ride” in the subject) the following information:

  • Your full, real name
  • Your blog or website, if any
  • Your company, school, or organization, if any
  • Your location, and closest airport
  • Any contributions you’ve made to MySQL or the MySQL community in the form of code, bug fixes, time, or otherwise
  • Why you feel you are the best or most qualified person for Proven Scaling to sponsor to MySQL Camp
  • Anything else you’d us like to know about yourself as we consider who to sponsor

However, you’d better hurry as you have only the next 60 hours or so to submit your proposal — no proposals will be accepted after midnight Pacific Daylight Time (GMT-8), Friday, October 20, 2006.

Good luck!

Update: Extended (and clarified) the deadline to midnight Friday instead of Thursday morning. Just to be clear: anyone is eligible, not just students.

a And, we’d encourage other companies attending or sponsoring MySQL Camp to do the same!

b If you live elsewhere, such as Europe, and you think you can get to the Bay Area fairly cheaply (lets say, under 1000 USD or so), or if you are just super qualified, feel free to send us your proposal anyway, and include how you propose to get to Mountain View cost effectively.

On Triggers, Stored Procedures, and Call Stacks

by Jeremy Cole on Sunday, October 1st, 2006 at 17:14:30 in MySQL, MySQL Patches, MySQL Tips

If you’re a frequent reader, you might have noticed that I’m on a roll contributing MySQL patches by now… there are many more to come. This is part of the reason that I founded Proven Scaling — to be able to spend my time solving interesting problems and making MySQL better. So what about triggers, stored procedures, and call stacks?

I’ve written a patch that implements three new functions in MySQL 5.01:

  • CALLER(level) — Returns the SQL statement in the call stack at level, where 0 is the level containing the call to CALLER() itself (which is nearly useless), and 1 and above are any stored procedure or trigger calls that got us here.
  • CALLER_DEPTH() — Returns the current depth of the call stack, not counting 0. CALLER(CALLER_DEPTH()) will always return the top-most level (i.e. the user-generated command).
  • CALLER_WS(separator, level) — Returns the entire call stack starting at level, with each level separated by separator.

What are these good for? Quite a bit! For a few examples:

  • Debugging — From any place in your stored procedure code, you can now find out how exactly you got there, no matter how complex the code is. In the future, this could be augmented by allowing you to see the values of the parameters at each step as well.
  • Auditing — This is much more interesting for most people, and while the current implementation isn’t perfect2, it’s getting a lot closer to what people need for auditing purposes. Using triggers and CALLER(), USER(), and NOW() you can get most of the auditing information you may need.

Now, for some examples of the output from these new functions:

Just a test of CALLER() and CALLER_DEPTH():

mysql> select caller(0);
+------------------+
| caller(0)        |
+------------------+
| select caller(0) |
+------------------+
1 row in set (0.00 sec)

mysql> select caller_depth();
+----------------+
| caller_depth() |
+----------------+
|              0 |
+----------------+
1 row in set (0.00 sec)

In order to get anything interesting, we need to generate some depth of the call stack:

delimiter //
create procedure stack() begin select caller_ws(' <called by> ', 1); end //
create procedure ts1() begin call stack(); end //
create procedure ts2() begin call ts1(); end //
create procedure ts3() begin call ts2(); end //

Now we can try it out for real:

mysql> call ts2()//
+------------------------------------------------------------+
| caller_ws(' <called by> ', 1)                              |
+------------------------------------------------------------+
| call stack() <called by> call ts1() <called by> call ts2() |
+------------------------------------------------------------+
1 row in set (0.00 sec)

As an example of what you can do inside stored procedures:

DROP PROCEDURE IF EXISTS test.dump_stack //
CREATE DEFINER='root'@'localhost' PROCEDURE test.dump_stack ()
BEGIN
  DECLARE i INT DEFAULT 1;
  DECLARE caller_depth INT DEFAULT 0;
  SET caller_depth = CALLER_DEPTH();

  CREATE TEMPORARY TABLE test_stack_t (
    depth INT NOT NULL,
    query TEXT,
    PRIMARY KEY (depth)
  );
  WHILE i <= caller_depth
  DO
    INSERT INTO test_stack_t (depth, query) VALUES (i, caller(i+1));
    SET i = i + 1;
  END WHILE;
  SELECT depth, query FROM test_stack_t ORDER BY depth DESC;
  DROP TEMPORARY TABLE test_stack_t;
END
//

We can then redefine stack() to call dump_stack() instead, so that we can use the same tsX() procedures above:

drop procedure if exists stack //
create procedure stack() begin call dump_stack(); end //

And the same test:

mysql> call ts2()//
+-------+--------------+
| depth | query        |
+-------+--------------+
|     3 | call ts2()   |
|     2 | call ts1()   |
|     1 | call stack() |
+-------+--------------+
3 rows in set (0.00 sec)

With a little additional magic, we can use CALLER() for auditing2:

DROP TABLE IF EXISTS test.audit //
CREATE TABLE test.audit (
  id INT NOT NULL auto_increment,
  ts DATETIME,
  user CHAR(64),
  call_stack TEXT,
  PRIMARY KEY (id),
  INDEX (ts),
  INDEX (user)
)
//

DROP PROCEDURE IF EXISTS test.audit //
CREATE DEFINER='root'@'localhost' PROCEDURE test.audit ()
BEGIN
  INSERT INTO test.audit (ts, user, call_stack)
  VALUES (now(), user(), caller_ws(' <called by> ', 2));
END
//

DROP TABLE IF EXISTS test.test //
CREATE TABLE test.test (
  c CHAR(50)
)
//

CREATE DEFINER='root'@'localhost' TRIGGER test_test_b_i
BEFORE INSERT ON test.test
FOR EACH ROW CALL test.audit()
//

CREATE DEFINER='root'@'localhost' PROCEDURE addtest(in in_c char(50))
BEGIN
  INSERT INTO test.test (c) VALUES (in_c);
END
//

And, let’s try it out:

mysql> INSERT INTO test.test (c) VALUES ('jeremy') //
Query OK, 1 row affected (0.01 sec)

mysql> SELECT * FROM test.audit //
+----+---------------------+---------------+---------------------------------------------+
| id | ts                  | user          | call_stack                                  |
+----+---------------------+---------------+---------------------------------------------+
|  4 | 2006-10-01 18:07:39 | root@dhcp-100 | INSERT INTO test.test (c) VALUES ('jeremy') |
+----+---------------------+---------------+---------------------------------------------+
1 row in set (0.00 sec)

mysql> CALL addtest('monty') //
Query OK, 1 row affected (0.00 sec)

mysql> SELECT * FROM test.audit //
+----+---------------------+---------------+---------------------------------------------------------------------------+
| id | ts                  | user          | call_stack                                                                |
+----+---------------------+---------------+---------------------------------------------------------------------------+
|  4 | 2006-10-01 18:07:39 | root@dhcp-100 | INSERT INTO test.test (c) VALUES ('jeremy')                               |
|  5 | 2006-10-01 18:11:26 | root@dhcp-100 | INSERT INTO test.test (c) VALUES (in_c) <called by> CALL addtest('monty') |
+----+---------------------+---------------+---------------------------------------------------------------------------+
2 rows in set (0.01 sec)

Let me know what you think!

1 Specifically mysql-5.0.25-nightly-20060823.

2 Currently, triggers are missing three important features to make them really useful for auditing: a way to find the call stack (which this patch addresses), the ability to have multiple triggers per type per table (currently MySQL only allows one trigger per table per type [before update, after insert, etc.]), and per-statement triggers (currently MySQL only has per-row triggers).