SQL and Modularity: Patterns, Anti-Patterns and the Kitchen Sink

There's a lot of importance placed on code re-use in the database development world. In traditional procedural programming languages, such as C or Fortran, the value of modular programming and its application to promoting code re-use is well known and understood. When SQL enters the picture, however, the situation becomes less clear, and there is less consensus on how best to apply the traditional concept of modularity.

This article will consider the concept of modularity, and how it may best be applied to SQL, from the perspective of 'patterns' and 'anti-patterns'. Here is a definition of these terms from Wikipedia, Anti-pattern:

An anti-pattern (or antipattern) is a pattern used in social or business operations or software engineering that may be commonly used but is ineffective and/or counterproductive in practice.[1][2]

The term was coined in 1995 by Andrew Koenig,[3] inspired by Gang of Four's book Design Patterns, which developed the concept of design patterns in the software field. The term was widely popularized three years later by the book AntiPatterns, which extended the use of the term beyond the field of software design and into general social interaction. According to the authors of the latter, there must be at least two key elements present to formally distinguish an actual anti-pattern from a simple bad habit, bad practice, or bad idea:

  • Some repeated pattern of action, process or structure that initially appears to be beneficial, but ultimately produces more bad consequences than beneficial results, and
  • An alternative solution exists that is clearly documented, proven in actual practice and repeatable

 

Procedural Modularity

Modularity starts from the idea that a complex design can generally be broken down into a set of less complex component modules that is easier to work with. In programming terms, a long main program would be broken down into smaller subroutines, with a much shorter main program that calls the subroutines.

From this starting point emerges the possibility of code re-use, whereby the decomposition into modules aims at identifying common logic that can be placed in generic modules and called in multiple places. A simple example of this would be an error-logging module in PL/SQL that would write any Oracle errors to a table along with call stack information, that could be called wherever such errors need to be trapped. This might be termed an error-logging pattern, and it's clear that this kind of code re-use can lead to simpler and more maintainable systems.

SQL Modularity: Design Patterns

Transactional APIs

The concept of transactions is important for modular design within a database application.

The Oracle manual, Oracle Database Concepts, defines a transaction thus:
A transaction is a logical unit of work that contains one or more SQL statements. A transaction is an atomic unit. The effects of all the SQL statements in a transaction can be either all committed (applied to the database) or all rolled back (undone from the database).

When a transaction needs to be performed in more than one place, then the code can be placed in a PL/SQL module, sometimes called a 'transactional API'. This is really just the database-specific version of standard modularity, and is obviously a good design pattern to follow.

Data Access Layers

Transactional APIs are often used to form a Data Access Layer (DAL) for front end applications written in languages such as Java. Where the front end requires a record set from the database, the APIs may return a reference cursor, which is essentially a pointer to the data, and avoids the overhead of passing the whole data set at once. The data access layer pattern has a number of important advantages:

  • performance is enhanced through reduced network traffic between application server and database
  • SQL operates in an efficient set-based fashion for retrieving data in batches
  • the PL/SQL language, highly integrated with SQL, is specifically designed for database processing
  • storing the database processing code in database packages promotes modularity and code re-use

It is considered best practice to use data access layers even for clients, such as Oracle Forms, that have an embedded PL/SQL engine.

Of course, in order to achieve these benefits the data access layer has to be correctly written, avoiding the anti-pattern pitfalls discussed later. In particular, 'kitchen sink' style APIs that return far more data than required, to promote re-use, must be avoided; different client programs requiring different data should have separate APIs.

Views

Database views have been available in Oracle SQL from the earliest versions and can be used to avoid duplicating a complex SQL query that might be needed in multiple places.

This approach could be seens as a special case of transactional modularity, where a single query is the transaction, and might be regarded as a design pattern for re-use of SQL statements.

Views may form an alternative kind of data access layer, typically used by reporting tools such as Business Objects, and may also be used in conjumction with an API-based layer, which is a common approach in Oracle Forms applications.

Within-SQL Modularity

SQL is essentially a declarative, rather than procedural language for retrieving (and updating etc.) data from relational databases. The original idea was that the programmer specifies the tables and columns where the data are stored, as well as how the tables are related through key values, but does not specify algorithms for retrieving the data: The retrieval algorithms are performed by the SQL engine 'under the covers'. There might therefore seem to be little scope for modularity within a SQL select statement. However, this is not quite true for a couple of reasons: First, logical paths have to be specified between the tables, and the same path may need to be specified multiple times from different starting points; for example, the path to billing and shipping addresses on a sales order would typically involve the same sequence of steps from different id columns; second, as SQL has evolved, procedural capabilities have been added, such as analytic functions and recursion.

In Oracle in-line views were introduced in v7.2, and are in a sense a first step in modularising an SQL statement, followed in v9.2 by the 'WITH' clause for subquery factoring. Analytic functions were introduced in v8i.

Here is an example based on Oracle's HR demo schema. Suppose we want a list of employees with their current and previous jobs (if any), the same for their manager, and a count of the number of subordinates they have. To get the previous jobs, we need to find the latest records in the job_history table for the employee and his manager separately. This can be done using subqueries, but that is inefficient and it is normally better to join to aggregation views that use the DENSE_RANK clause to allow the required previous records to be obtained in a single pass each. Similarly, the subordinate count could be done by a scalar subquery, but again performance would usually dictate the use of another aggregation view.

Here is a query using in-line views to achieve this:

SELECT emp.last_name || ', ' || emp.first_name name,
       job.job_title,
       job_p.job_title             job_title_prior,
       emp_m.last_name || ', ' || emp_m.first_name name_mgr,
       job_m.job_title             job_title_mgr,
       job_pm.job_title            job_title_mgr_prior,
       sub.n_sub
  FROM hr.employees                emp
  JOIN hr.jobs                     job
    ON job.job_id                  = emp.job_id
  LEFT JOIN (SELECT employee_id,
                    Max (job_id) KEEP (DENSE_RANK LAST ORDER BY end_date) job_id
               FROM hr.job_history
              GROUP BY employee_id
            )                      jhs
    ON jhs.employee_id             = emp.employee_id
  LEFT JOIN hr.jobs                job_p
    ON job_p.job_id                = jhs.job_id
  LEFT JOIN hr.employees           emp_m
    ON emp_m.employee_id           = emp.manager_id
  LEFT JOIN hr.jobs                job_m
    ON job_m.job_id                = emp_m.job_id
  LEFT JOIN (SELECT employee_id,
                    Max (job_id) KEEP (DENSE_RANK LAST ORDER BY end_date) job_id
               FROM hr.job_history
              GROUP BY employee_id
            )                      jhs_m
    ON jhs_m.employee_id           = emp.manager_id
  LEFT JOIN hr.jobs                job_pm
    ON job_pm.job_id               = jhs_m.job_id
  LEFT JOIN (SELECT manager_id,
                    Count(*)       n_sub
               FROM hr.employees
              GROUP BY manager_id
            )                      sub
    ON sub.manager_id              = emp.employee_id
 WHERE emp.department_id           = 30
 ORDER BY 1

Here is a query using subquery factors to achieve the same:

WITH jhs_f AS (
SELECT employee_id,
       Max (job_id) KEEP (DENSE_RANK LAST ORDER BY end_date) job_id
  FROM hr.job_history
 GROUP BY employee_id
), sub_f AS (
SELECT manager_id,
       Count(*)                    n_sub
  FROM hr.employees
 GROUP BY manager_id
)
SELECT emp.last_name || ', ' || emp.first_name name,
       job.job_title,
       job_p.job_title             job_title_prior,
       emp_m.last_name || ', ' || emp_m.first_name name_mgr,
       job_m.job_title             job_title_mgr,
       job_pm.job_title            job_title_mgr_prior,
       sub.n_sub
  FROM hr.employees                emp
  JOIN hr.jobs                     job
    ON job.job_id                  = emp.job_id
  LEFT JOIN jhs_f                  jhs
    ON jhs.employee_id             = emp.employee_id
  LEFT JOIN hr.jobs                job_p
    ON job_p.job_id                = jhs.job_id
  LEFT JOIN hr.employees           emp_m
    ON emp_m.employee_id           = emp.manager_id
  LEFT JOIN hr.jobs                job_m
    ON job_m.job_id                = emp_m.job_id
  LEFT JOIN jhs_f                  jhs_m
    ON jhs_m.employee_id           = emp.employee_id
  LEFT JOIN hr.jobs                job_pm
    ON job_pm.job_id               = jhs_m.job_id
  LEFT JOIN sub_f                  sub
    ON sub.manager_id              = emp.employee_id
 WHERE emp.department_id           = 30
 ORDER BY 1

The second query, although only a line shorter, could be said to be more modular in two ways:

  1. The more complex processing is placed at the beginning, prior to the main select, which now contains only simple joins. This might be said to parallel the procedural modularity practice of having a simple main program calling subroutines, and may help maintainability
  2. A single subquery factor replaces the two inline views for previous jobs, a more modular design, and one that may be more efficient for larger data sets since Oracle generally materialises subquery factors referenced multiple times

This approach might be regarded as a design pattern for modularity within individual SQL statements.

ANSI Join Syntax

The queries above are written using ANSI join syntax, introduced in v9. Oracle SQL originally used its own proprietary syntax, as shown below for the same query requirement:

WITH jhs_f AS (
SELECT employee_id,
       Max (job_id) KEEP (DENSE_RANK LAST ORDER BY end_date) job_id
  FROM hr.job_history
 GROUP BY employee_id
), sub_f AS (
SELECT manager_id,
       Count(*)                    n_sub
  FROM hr.employees
 GROUP BY manager_id
)
SELECT emp.last_name || ', ' || emp.first_name name,
       job.job_title,
       job_p.job_title             job_title_prior,
       emp_m.last_name || ', ' || emp_m.first_name name_mgr,
       job_m.job_title             job_title_mgr,
       job_pm.job_title            job_title_mgr_prior,
       sub.n_sub
  FROM hr.employees                emp,
       hr.jobs                     job,
       jhs_f                       jhs,
       hr.jobs                     job_p,
       hr.employees                emp_m,
       hr.jobs                     job_m,
       jhs_f                       jhs_m,
       hr.jobs                     job_pm,
       sub_f                       sub
 WHERE emp.department_id           = 30
   AND job.job_id                  = emp.job_id
   AND jhs.employee_id (+)         = emp.employee_id
   AND job_p.job_id (+)            = jhs.job_id
   AND emp_m.employee_id (+)       = emp.manager_id
   AND job_m.job_id (+)            = emp_m.job_id
   AND jhs_m.employee_id (+)       = emp.employee_id
   AND job_pm.job_id (+)           = jhs_m.job_id
   AND sub.manager_id (+)          = emp.employee_id
 ORDER BY 1

The tables are listed together, join clauses are in a single block not separated from constraints, and outer joins are specified using a (+) token against every column in the 'left' table. The outer join syntax leads to widespread bugs when developers miss the (+) from one of the columns, which silently converts the join to an inner join.

Some of the advantages of the newer syntax are:

  • Greater functionality is available, including full outer joining
  • Outer joining is much harder to get wrong
  • The syntax follows an ANSI standard
  • Locating the join conditions with the table being joined appears to be more modular and readable

ANSI join syntax might therefore be considered a good design pattern to follow.

SQL Modularity: Design Anti-patterns

everything-but-the-kitchen-sink-IDIOM

Here is a thread from Tom Kyte's AskTom forum dealing with an SQL anti-pattern that is unfortunately common, and strongly opposed by Tom Kyte: Considering SQL as a Service. The idea behind the anti-pattern seems to be to avoid repeating even simple table joins in SQL by hiding them in a special type of data access layer designed to be called within individual SQL statements. This results in over-complex 'kitchen-sink' SQL within the layer itself and performance problems in the 'client' SQL. In addition, complex PL/SQL involving object types and arrays tends to be needed to glue it all together; the approach thus achieves the opposite of its intended purpose - simplification - in the manner of a classic anti-pattern.

We'll illustrate the anti-pattern by extending the HR example used above and working through an example. First we'll use the PL/SQL packaged procedure variant, then look at an older form based on views.

APIs as SQL Building Blocks Anti-pattern

Let's suppose that we start from the idea that we should centralise the SQL for employee information in a re-useable API. We could take the SQL above and add in department name, address and manager information to make it more general. We might think initially of making the API a function taking an employee id as input and returning a record. But what if we needed the information for a list of employees? It would be inefficient to call a function for every record in the list, which might lead us to think of making the function take a list as input and return a list of records. We might therefore define object and array types, and a function with the following signature:

FUNCTION Emp_Info_List (p_emp_id_list SYS.ODCINumberList) RETURN emp_info_list_type;

[All code and output referenced is attached to this article.]

Now let's consider a scenario in which we have to provide an API for a web front end following the design pattern of returning a reference cursor. The data required are the following details for all employees in a given department:

  • employee name
  • manager name
  • list of subordinates

The output for department 30 would be:

NAME                      NAME_MGR               NAME_SUB
------------------------- ---------------------- -------------------
Baida, Shelli             Raphaely, Den
Colmenares, Karen         Raphaely, Den
Himuro, Guy               Raphaely, Den
Khoo, Alexander           Raphaely, Den
Raphaely, Den             King, Steven           Baida, Shelli
Raphaely, Den             King, Steven           Colmenares, Karen
Raphaely, Den             King, Steven           Himuro, Guy
Raphaely, Den             King, Steven           Khoo, Alexander
Raphaely, Den             King, Steven           Tobias, Sigal
Tobias, Sigal             Raphaely, Den

10 rows selected.

Here is a possible procedure implementation:

PROCEDURE Get_Mgr_Subs_KS (p_dept_id PLS_INTEGER, x_mgr_sub_cur OUT SYS_REFCURSOR) IS
  l_emp_id_list SYS.ODCINumberList;
BEGIN

  SELECT employee_id
    BULK COLLECT INTO l_emp_id_list
    FROM hr.employees
   WHERE department_id = p_dept_id;

  OPEN x_mgr_sub_cur FOR
  SELECT t.name,
         t.name_mgr,
         e.last_name || ', ' || e.first_name
    FROM TABLE (KSink_Emp.Emp_Info_List (l_emp_id_list)) t
    LEFT JOIN hr.employees e
      ON e.manager_id = t.employee_id
   ORDER BY 1, 2, 3;

END Get_Mgr_Subs_KS;

The first step is to get the list of employees for the department, which we then pass into the API, wrapped in the TABLE key word, and join the employees table to get the subordinates. Three SQL select statements are executed. You might argue that I have over-complicated this by having the API take a list of employees rather than the department id, but remember that in this design anti-pattern the API can't be designed for one specific caller, and the list input is more general. It is intended to cater for all calls for employee information so in practice such compromises will happen frequently.

We can compare this to an alternative implementation in which we simply join the tables required:

PROCEDURE Get_Mgr_Subs_SQL (p_dept_id PLS_INTEGER, x_mgr_sub_cur OUT SYS_REFCURSOR) IS
BEGIN

  OPEN x_mgr_sub_cur FOR
  SELECT e.last_name || ', ' || e.first_name,
         m.last_name || ', ' || m.first_name,
         s.last_name || ', ' || s.first_name
    FROM hr.employees e
    LEFT JOIN hr.employees m
      ON m.employee_id = e.manager_id
    LEFT JOIN hr.employees s
      ON s.manager_id = e.employee_id
   WHERE e.department_id = p_dept_id
   ORDER BY 1, 2, 3;

END Get_Mgr_Subs_SQL;

This joins three tables in one statement while the earlier procedure effectively makes a join through PL/SQL using an array, which is arguably slightly more complicated. In any case the real problems become apparent when you compare the execution plans. I have written a test driver program that calls each of the APIs and loops over the returned cursor.

Taking the plan for the straight SQL implementation first:

-------------------------------------------------------------------------------------------------------------------------------------------------
| Id  | Operation                              | Name              | Starts | E-Rows | A-Rows |   A-Time   | Buffers |  OMem |  1Mem | Used-Mem |
-------------------------------------------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT                       |                   |      1 |        |     10 |00:00:00.01 |      14 |       |       |          |
|   1 |  SORT ORDER BY                         |                   |      1 |     35 |     10 |00:00:00.01 |      14 |  2048 |  2048 | 2048  (0)|
|   2 |   NESTED LOOPS OUTER                   |                   |      1 |     35 |     10 |00:00:00.01 |      14 |       |       |          |
|*  3 |    HASH JOIN OUTER                     |                   |      1 |      6 |      6 |00:00:00.01 |      10 |  1281K|  1281K|  544K (0)|
|   4 |     TABLE ACCESS BY INDEX ROWID BATCHED| EMPLOYEES         |      1 |      6 |      6 |00:00:00.01 |       2 |       |       |          |
|*  5 |      INDEX RANGE SCAN                  | EMP_DEPARTMENT_IX |      1 |      6 |      6 |00:00:00.01 |       1 |       |       |          |
|   6 |     VIEW                               | index$_join$_002  |      1 |    107 |    107 |00:00:00.01 |       8 |       |       |          |
|*  7 |      HASH JOIN                         |                   |      1 |        |    107 |00:00:00.01 |       8 |  1245K|  1245K| 1439K (0)|
|   8 |       INDEX FAST FULL SCAN             | EMP_NAME_IX       |      1 |    107 |    107 |00:00:00.01 |       4 |       |       |          |
|   9 |       INDEX FAST FULL SCAN             | EMP_EMP_ID_PK     |      1 |    107 |    107 |00:00:00.01 |       4 |       |       |          |
|  10 |    TABLE ACCESS BY INDEX ROWID BATCHED | EMPLOYEES         |      6 |      6 |      5 |00:00:00.01 |       4 |       |       |          |
|* 11 |     INDEX RANGE SCAN                   | EMP_MANAGER_IX    |      6 |      6 |      5 |00:00:00.01 |       3 |       |       |          |
-------------------------------------------------------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   3 - access("M"."EMPLOYEE_ID"="E"."MANAGER_ID")
   5 - access("E"."DEPARTMENT_ID"=:B1)
   7 - access(ROWID=ROWID)
  11 - access("S"."MANAGER_ID"="E"."EMPLOYEE_ID")

This is a relatively simple plan for the single SQL statement, with 14 buffers read.

For the anti-pattern version there are thee SQL select statements, but we'll ignore the plan for initial bulk collect SQL and consider the other two execution plans. First, the client API SQL:

---------------------------------------------------------------------------------------------------------------------------------------------------------------
| Id  | Operation                           | Name             | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  | Writes |  OMem |  1Mem | Used-Mem |
---------------------------------------------------------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT                    |                  |      1 |        |     10 |00:00:00.14 |    1195 |     41 |      1 |       |       |          |
|   1 |  SORT ORDER BY                      |                  |      1 |  48554 |     10 |00:00:00.14 |    1195 |     41 |      1 |  2048 |  2048 | 2048  (0)|
|*  2 |   HASH JOIN RIGHT OUTER             |                  |      1 |  48554 |     10 |00:00:00.14 |    1195 |     41 |      1 |  1368K|  1368K| 1322K (0)|
|   3 |    VIEW                             | index$_join$_002 |      1 |    107 |    106 |00:00:00.02 |       8 |     12 |      0 |       |       |          |
|*  4 |     HASH JOIN                       |                  |      1 |        |    106 |00:00:00.02 |       8 |     12 |      0 |  1519K|  1519K| 1575K (0)|
|   5 |      INDEX FAST FULL SCAN           | EMP_MANAGER_IX   |      1 |    107 |    106 |00:00:00.01 |       4 |      6 |      0 |       |       |          |
|   6 |      INDEX FAST FULL SCAN           | EMP_NAME_IX      |      1 |    107 |    107 |00:00:00.01 |       4 |      6 |      0 |       |       |          |
|   7 |    COLLECTION ITERATOR PICKLER FETCH| EMP_INFO_LIST    |      1 |   8168 |      6 |00:00:00.13 |    1187 |     29 |      1 |       |       |          |
---------------------------------------------------------------------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   2 - access("E"."MANAGER_ID"=SYS_OP_ATG(VALUE(KOKBF$),1,2,2))
   4 - access(ROWID=ROWID)

Note the extreme inaccuracy of the cardinality estimates at steps 1 and 2, which originate in the step 7 estimate of 8168, which is a database-level default for an array function call. This is exposing the general problem that joining to arrays prevents accurate cardinality estimates. A total of 1195 buffers were read.

Next, the plan for the inner API SQL:

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| Id  | Operation                                       | Name                      | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  | Writes |  OMem |  1Mem | Used-Mem |
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT                                |                           |      1 |        |      6 |00:00:00.01 |      94 |      1 |      1 |       |       |          |
|   1 |  TEMP TABLE TRANSFORMATION                      |                           |      1 |        |      6 |00:00:00.01 |      94 |      1 |      1 |       |       |          |
|   2 |   LOAD AS SELECT                                |                           |      1 |        |      0 |00:00:00.01 |       6 |      0 |      1 |  1036K|  1036K|          |
|   3 |    SORT GROUP BY NOSORT                         |                           |      1 |      7 |      7 |00:00:00.01 |       2 |      0 |      0 |       |       |          |
|   4 |     TABLE ACCESS BY INDEX ROWID                 | JOB_HISTORY               |      1 |     10 |     10 |00:00:00.01 |       2 |      0 |      0 |       |       |          |
|   5 |      INDEX FULL SCAN                            | JHIST_EMP_ID_ST_DATE_PK   |      1 |     10 |     10 |00:00:00.01 |       1 |      0 |      0 |       |       |          |
|*  6 |   HASH JOIN OUTER                               |                           |      1 |      6 |      6 |00:00:00.01 |      85 |      1 |      0 |   735K|   735K|  506K (0)|
|*  7 |    HASH JOIN OUTER                              |                           |      1 |      6 |      6 |00:00:00.01 |      78 |      1 |      0 |   736K|   736K|  891K (0)|
|*  8 |     HASH JOIN OUTER                             |                           |      1 |      6 |      6 |00:00:00.01 |      75 |      1 |      0 |   737K|   737K|  550K (0)|
|*  9 |      HASH JOIN OUTER                            |                           |      1 |      6 |      6 |00:00:00.01 |      68 |      1 |      0 |   739K|   739K|  883K (0)|
|* 10 |       HASH JOIN OUTER                           |                           |      1 |      6 |      6 |00:00:00.01 |      62 |      0 |      0 |   740K|   740K|  903K (0)|
|* 11 |        HASH JOIN OUTER                          |                           |      1 |      6 |      6 |00:00:00.01 |      55 |      0 |      0 |   746K|   746K|  541K (0)|
|* 12 |         HASH JOIN OUTER                         |                           |      1 |      6 |      6 |00:00:00.01 |      48 |      0 |      0 |   754K|   754K|  534K (0)|
|* 13 |          HASH JOIN OUTER                        |                           |      1 |      6 |      6 |00:00:00.01 |      41 |      0 |      0 |   766K|   766K|  418K (0)|
|* 14 |           HASH JOIN OUTER                       |                           |      1 |      6 |      6 |00:00:00.01 |      33 |      0 |      0 |   773K|   773K|  414K (0)|
|  15 |            NESTED LOOPS OUTER                   |                           |      1 |      6 |      6 |00:00:00.01 |      26 |      0 |      0 |       |       |          |
|* 16 |             HASH JOIN OUTER                     |                           |      1 |      6 |      6 |00:00:00.01 |      23 |      0 |      0 |   833K|   833K|  414K (0)|
|* 17 |              HASH JOIN OUTER                    |                           |      1 |      6 |      6 |00:00:00.01 |      16 |      0 |      0 |   876K|   876K|  415K (0)|
|* 18 |               HASH JOIN                         |                           |      1 |      6 |      6 |00:00:00.01 |       9 |      0 |      0 |   905K|   905K| 1259K (0)|
|  19 |                MERGE JOIN                       |                           |      1 |    107 |    107 |00:00:00.01 |       9 |      0 |      0 |       |       |          |
|  20 |                 TABLE ACCESS BY INDEX ROWID     | JOBS                      |      1 |     19 |     19 |00:00:00.01 |       2 |      0 |      0 |       |       |          |
|  21 |                  INDEX FULL SCAN                | JOB_ID_PK                 |      1 |     19 |     19 |00:00:00.01 |       1 |      0 |      0 |       |       |          |
|* 22 |                 SORT JOIN                       |                           |     19 |    107 |    107 |00:00:00.01 |       7 |      0 |      0 | 18432 | 18432 |16384  (0)|
|  23 |                  TABLE ACCESS FULL              | EMPLOYEES                 |      1 |    107 |    107 |00:00:00.01 |       7 |      0 |      0 |       |       |          |
|  24 |                COLLECTION ITERATOR PICKLER FETCH|                           |      1 |      6 |      6 |00:00:00.01 |       0 |      0 |      0 |       |       |          |
|  25 |               TABLE ACCESS FULL                 | DEPARTMENTS               |      1 |     27 |     27 |00:00:00.01 |       7 |      0 |      0 |       |       |          |
|  26 |              TABLE ACCESS FULL                  | LOCATIONS                 |      1 |     23 |     23 |00:00:00.01 |       7 |      0 |      0 |       |       |          |
|* 27 |             INDEX UNIQUE SCAN                   | COUNTRY_C_ID_PK           |      6 |      1 |      6 |00:00:00.01 |       3 |      0 |      0 |       |       |          |
|  28 |            TABLE ACCESS FULL                    | REGIONS                   |      1 |      4 |      4 |00:00:00.01 |       7 |      0 |      0 |       |       |          |
|  29 |           VIEW                                  | index$_join$_024          |      1 |    107 |    107 |00:00:00.01 |       8 |      0 |      0 |       |       |          |
|* 30 |            HASH JOIN                            |                           |      1 |        |    107 |00:00:00.01 |       8 |      0 |      0 |  1245K|  1245K| 1550K (0)|
|  31 |             INDEX FAST FULL SCAN                | EMP_NAME_IX               |      1 |    107 |    107 |00:00:00.01 |       4 |      0 |      0 |       |       |          |
|  32 |             INDEX FAST FULL SCAN                | EMP_EMP_ID_PK             |      1 |    107 |    107 |00:00:00.01 |       4 |      0 |      0 |       |       |          |
|  33 |          TABLE ACCESS FULL                      | EMPLOYEES                 |      1 |    107 |    107 |00:00:00.01 |       7 |      0 |      0 |       |       |          |
|  34 |         TABLE ACCESS FULL                       | JOBS                      |      1 |     19 |     19 |00:00:00.01 |       7 |      0 |      0 |       |       |          |
|  35 |        VIEW                                     |                           |      1 |     18 |     19 |00:00:00.01 |       7 |      0 |      0 |       |       |          |
|  36 |         HASH GROUP BY                           |                           |      1 |     18 |     19 |00:00:00.01 |       7 |      0 |      0 |  1558K|  1558K| 1185K (0)|
|  37 |          TABLE ACCESS FULL                      | EMPLOYEES                 |      1 |    107 |    107 |00:00:00.01 |       7 |      0 |      0 |       |       |          |
|  38 |       VIEW                                      |                           |      1 |      7 |      7 |00:00:00.01 |       6 |      1 |      0 |       |       |          |
|  39 |        TABLE ACCESS FULL                        | SYS_TEMP_0FD9D6604_5813E5 |      1 |      7 |      7 |00:00:00.01 |       6 |      1 |      0 |       |       |          |
|  40 |      TABLE ACCESS FULL                          | JOBS                      |      1 |     19 |     19 |00:00:00.01 |       7 |      0 |      0 |       |       |          |
|  41 |     VIEW                                        |                           |      1 |      7 |      7 |00:00:00.01 |       3 |      0 |      0 |       |       |          |
|  42 |      TABLE ACCESS FULL                          | SYS_TEMP_0FD9D6604_5813E5 |      1 |      7 |      7 |00:00:00.01 |       3 |      0 |      0 |       |       |          |
|  43 |    TABLE ACCESS FULL                            | JOBS                      |      1 |     19 |     19 |00:00:00.01 |       7 |      0 |      0 |       |       |          |
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   6 - access("JOB_P"."JOB_ID"="JHS"."JOB_ID")
   7 - access("JHS"."EMPLOYEE_ID"="EMP"."EMPLOYEE_ID")
   8 - access("JOB_MP"."JOB_ID"="JHS_M"."JOB_ID")
   9 - access("JHS_M"."EMPLOYEE_ID"="EMP"."EMPLOYEE_ID")
  10 - access("SUB"."MANAGER_ID"="EMP"."EMPLOYEE_ID")
  11 - access("JOB_M"."JOB_ID"="EMP_M"."JOB_ID")
  12 - access("EMP_M"."EMPLOYEE_ID"="EMP"."MANAGER_ID")
  13 - access("EMP_DM"."EMPLOYEE_ID"="DEP"."MANAGER_ID")
  14 - access("REG"."REGION_ID"="COU"."REGION_ID")
  16 - access("LOC"."LOCATION_ID"="DEP"."LOCATION_ID")
  17 - access("DEP"."DEPARTMENT_ID"="EMP"."DEPARTMENT_ID")
  18 - access("EMP"."EMPLOYEE_ID"=VALUE(KOKBF$))
  22 - access("JOB"."JOB_ID"="EMP"."JOB_ID")
       filter("JOB"."JOB_ID"="EMP"."JOB_ID")
  27 - access("COU"."COUNTRY_ID"="LOC"."COUNTRY_ID")
  30 - access(ROWID=ROWID)

This is pretty complex. We can't pretend to do a real performance analysis on such a small database (107 employees), but the potential for performance problems in real cases is clear. It's important to understand that any performance analysis on the client API has to take into account not just the client code, but the full complexity of the centralised SQL, so any apparent simplification from using it is to a great extent illusory.

Database Inter-Schema Data Access Layer Anti-pattern

There is a variant of the anti-pattern above in which data access layers are used to retrieve data across schema boundaries. This variant suffers from exactly the same problems as the first of course, and should equally be avoided.

Views as SQL Building Blocks Anti-Pattern

The same ideas as are behind the APIs as SQL Building Blocks Anti-Pattern can also be implemented through views, and in fact this variant form of the anti-pattern has been around longer I think. We can illustrate it on the same example, by creating a view instead of the central API cursor.

CREATE OR REPLACE VIEW emp_ks_v (
       employee_id,
       name,
       job_title,
       job_title_p,
       name_mgr,
       job_title_mgr,
       job_title_mgr_p,
       n_sub,
       department_id,
       department_name,
       name_d_mgr,
       street_address,
       country_name,
       region_name) AS
WITH jhs_f AS (
SELECT employee_id,
     Max (job_id) KEEP (DENSE_RANK LAST ORDER BY end_date) job_id
  FROM hr.job_history
 GROUP BY employee_id
), sub AS (
SELECT manager_id,
       Count(*)                  n_sub
  FROM hr.employees
 GROUP BY manager_id
)
SELECT emp.employee_id,
       emp.last_name || ', ' || emp.first_name,
       job.job_title,
       job_p.job_title,
       emp_m.last_name || ', ' || emp_m.first_name,
       job_m.job_title,
       job_mp.job_title,
       sub.n_sub,
       dep.department_id,
       dep.department_name,
       emp_dm.last_name || ', ' || emp_dm.first_name,
       loc.street_address,
       cou.country_name,
       reg.region_name
  FROM hr.employees              emp
  JOIN hr.jobs                   job
    ON job.job_id                = emp.job_id
  LEFT JOIN jhs_f                jhs
    ON jhs.employee_id           = emp.employee_id
  LEFT JOIN hr.jobs              job_p
    ON job_p.job_id              = jhs.job_id
  LEFT JOIN hr.employees         emp_m
    ON emp_m.employee_id         = emp.manager_id
  LEFT JOIN hr.jobs              job_m
    ON job_m.job_id              = emp_m.job_id
  LEFT JOIN jhs_f                jhs_m
    ON jhs_m.employee_id         = emp.employee_id
  LEFT JOIN hr.jobs              job_mp
    ON job_mp.job_id             = jhs_m.job_id
  LEFT JOIN sub
    ON sub.manager_id            = emp.employee_id
  LEFT JOIN hr.departments       dep
    ON dep.department_id         = emp.department_id
  LEFT JOIN hr.employees         emp_dm
    ON emp_dm.employee_id        = dep.manager_id
  LEFT JOIN hr.locations         loc
    ON loc.location_id           = dep.location_id
  LEFT JOIN hr.countries         cou
    ON cou.country_id            = loc.country_id
  LEFT JOIN hr.regions           reg
    ON reg.region_id             = cou.region_id

The view can then be called to get the employee details for example for a given department, thus:

SELECT t.name,
       t.name_mgr,
       CASE WHEN e.last_name IS NOT NULL THEN e.last_name || ', ' || e.first_name END name_sub
  FROM emp_ks_v t
  LEFT JOIN hr.employees e
    ON e.manager_id = t.employee_id
 WHERE t.department_id = 30
 ORDER BY 1, 2, 3

This is actually quite a lot better than the API-based approach as it's much simpler, avoiding the need for object arrays, and allowing use simply by joining. Let's look at the execution plan though (we'll just run the query rather than put it into a client API returning a reference cursor):

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| Id  | Operation                                    | Name                      | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  | Writes |  OMem |  1Mem | Used-Mem |
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT                             |                           |      1 |        |     10 |00:00:00.01 |      42 |      1 |      1 |       |       |          |
|   1 |  SORT ORDER BY                               |                           |      1 |     35 |     10 |00:00:00.01 |      42 |      1 |      1 |  2048 |  2048 | 2048  (0)|
|   2 |   NESTED LOOPS OUTER                         |                           |      1 |     35 |     10 |00:00:00.01 |      42 |      1 |      1 |       |       |          |
|   3 |    VIEW                                      | EMP_KS_V                  |      1 |      6 |      6 |00:00:00.01 |      38 |      1 |      1 |       |       |          |
|   4 |     TEMP TABLE TRANSFORMATION                |                           |      1 |        |      6 |00:00:00.01 |      38 |      1 |      1 |       |       |          |
|   5 |      LOAD AS SELECT                          |                           |      1 |        |      0 |00:00:00.01 |       6 |      0 |      1 |  1036K|  1036K|          |
|   6 |       SORT GROUP BY NOSORT                   |                           |      1 |      7 |      7 |00:00:00.01 |       2 |      0 |      0 |       |       |          |
|   7 |        TABLE ACCESS BY INDEX ROWID           | JOB_HISTORY               |      1 |     10 |     10 |00:00:00.01 |       2 |      0 |      0 |       |       |          |
|   8 |         INDEX FULL SCAN                      | JHIST_EMP_ID_ST_DATE_PK   |      1 |     10 |     10 |00:00:00.01 |       1 |      0 |      0 |       |       |          |
|*  9 |      HASH JOIN OUTER                         |                           |      1 |      6 |      6 |00:00:00.01 |      29 |      1 |      0 |   883K|   883K|  517K (0)|
|* 10 |       HASH JOIN OUTER                        |                           |      1 |      6 |      6 |00:00:00.01 |      21 |      1 |      0 |   890K|   890K|  857K (0)|
|* 11 |        HASH JOIN OUTER                       |                           |      1 |      6 |      6 |00:00:00.01 |      14 |      1 |      0 |   895K|   895K|  886K (0)|
|* 12 |         HASH JOIN OUTER                      |                           |      1 |      6 |      6 |00:00:00.01 |      11 |      1 |      0 |   905K|   905K|  893K (0)|
|  13 |          NESTED LOOPS                        |                           |      1 |      6 |      6 |00:00:00.01 |       5 |      0 |      0 |       |       |          |
|  14 |           NESTED LOOPS OUTER                 |                           |      1 |      1 |      1 |00:00:00.01 |       3 |      0 |      0 |       |       |          |
|  15 |            TABLE ACCESS BY INDEX ROWID       | DEPARTMENTS               |      1 |      1 |      1 |00:00:00.01 |       2 |      0 |      0 |       |       |          |
|* 16 |             INDEX UNIQUE SCAN                | DEPT_ID_PK                |      1 |      1 |      1 |00:00:00.01 |       1 |      0 |      0 |       |       |          |
|* 17 |            INDEX UNIQUE SCAN                 | LOC_ID_PK                 |      1 |      1 |      1 |00:00:00.01 |       1 |      0 |      0 |       |       |          |
|  18 |           TABLE ACCESS BY INDEX ROWID BATCHED| EMPLOYEES                 |      1 |      6 |      6 |00:00:00.01 |       2 |      0 |      0 |       |       |          |
|* 19 |            INDEX RANGE SCAN                  | EMP_DEPARTMENT_IX         |      1 |      6 |      6 |00:00:00.01 |       1 |      0 |      0 |       |       |          |
|  20 |          VIEW                                |                           |      1 |      7 |      7 |00:00:00.01 |       6 |      1 |      0 |       |       |          |
|  21 |           TABLE ACCESS FULL                  | SYS_TEMP_0FD9D6606_5813E5 |      1 |      7 |      7 |00:00:00.01 |       6 |      1 |      0 |       |       |          |
|  22 |         VIEW                                 |                           |      1 |      7 |      7 |00:00:00.01 |       3 |      0 |      0 |       |       |          |
|  23 |          TABLE ACCESS FULL                   | SYS_TEMP_0FD9D6606_5813E5 |      1 |      7 |      7 |00:00:00.01 |       3 |      0 |      0 |       |       |          |
|  24 |        VIEW                                  |                           |      1 |     18 |     19 |00:00:00.01 |       7 |      0 |      0 |       |       |          |
|  25 |         HASH GROUP BY                        |                           |      1 |     18 |     19 |00:00:00.01 |       7 |      0 |      0 |  1558K|  1558K| 1211K (0)|
|  26 |          TABLE ACCESS FULL                   | EMPLOYEES                 |      1 |    107 |    107 |00:00:00.01 |       7 |      0 |      0 |       |       |          |
|  27 |       VIEW                                   | index$_join$_013          |      1 |    107 |    107 |00:00:00.01 |       8 |      0 |      0 |       |       |          |
|* 28 |        HASH JOIN                             |                           |      1 |        |    107 |00:00:00.01 |       8 |      0 |      0 |  1245K|  1245K| 1410K (0)|
|  29 |         INDEX FAST FULL SCAN                 | EMP_NAME_IX               |      1 |    107 |    107 |00:00:00.01 |       4 |      0 |      0 |       |       |          |
|  30 |         INDEX FAST FULL SCAN                 | EMP_EMP_ID_PK             |      1 |    107 |    107 |00:00:00.01 |       4 |      0 |      0 |       |       |          |
|  31 |    TABLE ACCESS BY INDEX ROWID BATCHED       | EMPLOYEES                 |      6 |      6 |      5 |00:00:00.01 |       4 |      0 |      0 |       |       |          |
|* 32 |     INDEX RANGE SCAN                         | EMP_MANAGER_IX            |      6 |      6 |      5 |00:00:00.01 |       3 |      0 |      0 |       |       |          |
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   9 - access("EMP_M"."EMPLOYEE_ID"="EMP"."MANAGER_ID")
  10 - access("SUB"."MANAGER_ID"="EMP"."EMPLOYEE_ID")
  11 - access("JHS_M"."EMPLOYEE_ID"="EMP"."EMPLOYEE_ID")
  12 - access("JHS"."EMPLOYEE_ID"="EMP"."EMPLOYEE_ID")
  16 - access("DEP"."DEPARTMENT_ID"=30)
  17 - access("LOC"."LOCATION_ID"="DEP"."LOCATION_ID")
  19 - access("EMP"."DEPARTMENT_ID"=30)
  28 - access(ROWID=ROWID)
  32 - access("E"."MANAGER_ID"="T"."EMPLOYEE_ID")

There is only one SQL statement, and the plan is better too, with better cardinality estimates owing to the absence of array processing, and with 42 buffers read. It's still a bad idea though because the client caller would be executing far more complex SQL than is required and as before performance analysis on the client requires the full complexity of the 'centralised' SQL to be included. Using complex views as SQL building blocks is generally considered to have poor performance characteristics.

SQL Modularity: Other Design Options

Splitting Up Long SQL Statements

Oracle's Cost Based Optimiser (CBO) has been greatly enhanced since it's introduction in v7, but remains imperfect. The combinatorial nature of the problem that it tries to solve suggests that there will always be larger queries where it makes a bad choice of plan. In some cases splitting a large query into smaller ones and using temporary tables to join them can give better performance. This may arise from new indexing options for the CBO, or by dynamic sampling capabilities on the temporary tables, or just from the CBO algorithms happening to work better on the divided queries.

It's important to understand though that any such splitting should be done purely on performance grounds: the splitting increases the code complexity and breaking a sequence of declarative joins into several subsequences is not comparable with standard modularisation of programs into subprograms.

Long SQL statements are not necessarily problematic, but obviously should only be as long as necessary, and avoiding the design anti-patterns mentioned helps to ensure this.

Simple Views

Simple views, without joins, are often used in areas such as access control; for example Oracle Applications multi-org features (upto release 11) generally involve transactional tables being referenced by such simple views. These do not cause the performance problems seen with the complex building-block views anti-pattern.

Data Access Layers for Back-end Programs

It is quite possible to use a similar approach for database programs as for front-end programs in terms of a data access layer. One could adopt a standard that stand-alone database PL/SQL programs should access data through packaged APIs rather than directly. This obviously does not have the performance or language advantages of the design pattern in relation to Java front-ends for example, but it may in some circumstances be a preferred method of code organisation. It is therefore not an anti-pattern of the kind we have considered - as long as it is for stand-alone programs only.

Conclusions

The main aim of this article has been to distinguish between good approaches to modularity in SQL (patterns) and bad ones (anti-patterns) based on personal experience of seeing both types applied.

  • The Data Access Layer design pattern is an excellent approach for client applications developed in Java, .net etc. to access a database
  • Using Data Access Layers for internal access within a database is a classic anti-pattern leading to overcomplication and performance problems
  • A good design pattern used in an inappropriate context can become an anti-pattern

SQL Modularity Code






 

SQL for the Travelling Salesman Problem

'The travelling salesman problem (TSP) or travelling salesperson problem asks the following question: Given a list of cities and the distances between each pair of cities, what is the shortest possible route that visits each city exactly once and returns to the origin city? It is an NP-hard problem in combinatorial optimization, important in operations research and theoretical computer science.' - Travelling salesman problem.

['or travelling salesperson problem' - :)]

I posted a couple of articles recently on approximate solution methods for 'hard' combinatorial problems using SQL, SQL for the Balanced Number Partitioning Problem and SQL for the Fantasy Football Knapsack Problem, and I wondered whether similar techniques could be applied to TSP. This article provides my answer.

Updated, 20 February 2016: Added attachment with the input data files, DDL and raw results at the end.

Test Problems

I used two test problems, the first being a simple constructed example of five towns arranged in an M-shape. The second is a more realistic problem based on 312 towns in USA and Canada, the data for which I found on a university web site.

Test Problem 1: Emland

The first problem is small enough to verify the solutions manually.

The Short Route
TSP, v1.0 - Short

The Long Route
TSP, v1.0 - Long
Test Problem 2: USCA312

I got the second problem here, City Distance Datasets, described thus:
'USCA312 describes 312 cities in the US and Canada. Distances between the city are computed from latitude and longitude, not from road mileage'

I took the distances based on the x-y values given, using the simple Euclidean distance formula. I am aware that this will not give accurate distances, but it seemed the simplest approach as the provided distances were in a format not so easy to load into a database, and I am interested only in the technical aspects.

SQL Solution with Recursive Subquery Factoring

SQL

WITH count_towns AS (
SELECT Count(*) n_towns FROM towns
), dist_from_root AS (/* XTSP  */
SELECT a, b, dst, Row_Number () OVER (ORDER BY :SIGN * dst) rnk_by_dst, Count(DISTINCT a) OVER () + 1 n_towns
  FROM distances
 WHERE b > a
),  rsf (root, root_leg, path_rnk, nxt_id, lev, tot_price, path, n_towns) AS (
SELECT a, a || ' -> ' || b, 0, d.b, 2, d.dst, 
       CAST ('|' || LPad (d.a, 3, '0') || '|' || LPad (d.b, 3, '0') AS VARCHAR2(4000)) path,
       d.n_towns
  FROM dist_from_root d
 WHERE d.rnk_by_dst <= :KEEP_NUM_ROOT
 UNION ALL
SELECT r.root,
       r.root_leg,
       Row_Number() OVER (PARTITION BY r.root_leg ORDER BY :SIGN * (r.tot_price + d.dst)),
       d.b,
       r.lev + 1,
       r.tot_price + d.dst,
       r.path || '|' || LPad (d.b, 3, '0'),
       r.n_towns
  FROM rsf r
  JOIN distances d
    ON d.a = r.nxt_id
   AND r.path NOT LIKE '%' || '|' || LPad (d.b, 3, '0') || '%'
 WHERE r.path_rnk <= :KEEP_NUM
), circuits AS (
SELECT r.root_leg,
       Row_Number() OVER (PARTITION BY r.root_leg ORDER BY :SIGN * (r.tot_price + d.dst)) path_rnk,
       r.tot_price + d.dst tot_price,
       r.path || '|' || LPad (d.b, 3, '0') path
  FROM rsf r
  JOIN distances d
    ON d.a = r.nxt_id
   AND d.b = r.root
 WHERE r.lev = r.n_towns
   AND r.path_rnk <= :KEEP_NUM
), top_n_paths AS (
SELECT root_leg,
       tot_price,
       path,
       path_rnk,
       town_index
  FROM circuits
  CROSS JOIN (SELECT LEVEL town_index FROM count_towns c CONNECT BY LEVEL <= c.n_towns + 1)
 WHERE path_rnk <= :KEEP_NUM
), top_n_sets AS (
SELECT root_leg,
       tot_price,
       path,
       path_rnk,
       town_index,
       To_Number (Substr (path, (town_index - 1) * 4 + 2, 3)) town_id,
       Lag (To_Number (Substr (path, (town_index - 1) * 4 + 2, 3))) OVER (PARTITION BY root_leg, path_rnk ORDER BY town_index) town_id_prior
  FROM top_n_paths
)
SELECT /*+ GATHER_PLAN_STATISTICS */
       top.root_leg,
       top.path_rnk,
       Round (top.tot_price, 2) tot_dist,
       top.town_id,
       twn.name,
       Round (dst.dst, 2) leg_dist,
       Round (Sum (dst.dst) OVER (PARTITION BY root_leg, path_rnk ORDER BY town_index), 2) cum_dist
  FROM top_n_sets top
  JOIN towns twn
    ON twn.id = top.town_id
  LEFT JOIN distances dst
    ON dst.a = top.town_id_prior
   AND dst.b = top.town_id
ORDER BY top.root_leg, top.path_rnk, top.town_index

How It Works

Bind Variables
==============
SIGN - 1 means minimise distance; -1, maximise it
KEEP_NUM_ROOT - number of best anchor records (root legs) to retain (throughout)
KEEP_NUM - number of best records from previous iteration to retain, partitioning by root leg

The solution approach is based on the idea I had in my last article whereby recursive subquery factoring is at the heart of the method, and analytic row numbering is used to reduce searching to manageable proportions at each iteration. The bind variables control the level of searching desired.

  • count_towns subquery - simply counts the total number of towns
  • dist_from_root subquery - gets the distance between each pair of towns, without duplication, and ranks the pairs by distance
  • rsf subquery, anchor branch - selects the KEEP_NUM_ROOT best legs as roots for the recursion
  • rsf subquery, recursive branch - joins up to KEEP_NUM best records from last iteration all possible town via distances table, using pattern matching to exclude towns already visited
  • The recursive branch stores the towns visited in a delimited string
  • Analytic funtion Row_Number is used to rank the recursive branch result set at each iteration, partitioning by root leg for use in the next iteration
  • circuits subquery - completes the circuit by joining the root leg via the distances table, doing the same constraint by rank and analytic ranking as the recursive branch
  • top_n_paths subquery - uses a standard row generator subquery to generate indexed rows for the number of towns in the final path (my where clause looks redundant as I write)
  • top_n_sets subquery - extracts the town id from the path using the index derived in the previous subquery, and gets the previous town id using analytic Lag
  • Main query - joins towns table to get the town name and distances table for leg distances

Results

Test Problem 1: Emland

I used keep values of 5 and 5 and solved the shortest routes problem in 0.14 seconds, and the longest in 0.16 seconds.

Shortest Routes
===============
ROOT_LEG       PATH_RNK   TOT_DIST    TOWN_ID NAME                             LEG_DIST   CUM_DIST
------------ ---------- ---------- ---------- ------------------------------ ---------- ----------
1 -> 2                1      10.94          1 Left Floor
                                            2 Left Peak                            2.24       2.24
                                            4 Right Peak                              2       4.24
                                            5 Right Floor                          2.24       6.47
                                            3 Midfield                             2.24       8.71
                                            1 Left Floor                           2.24      10.94
                      2       11.3          1 Left Floor
                                            2 Left Peak                            2.24       2.24
                                            3 Midfield                             1.41       3.65
                                            4 Right Peak                           1.41       5.06
                                            5 Right Floor                          2.24        7.3
                                            1 Left Floor                              4       11.3
                      3      11.73          1 Left Floor
                                            2 Left Peak                            2.24       2.24
                                            5 Right Floor                          3.61       5.84
                                            4 Right Peak                           2.24       8.08
                                            3 Midfield                             1.41       9.49
                                            1 Left Floor                           2.24      11.73
                      4      11.73          1 Left Floor
                                            2 Left Peak                            2.24       2.24
                                            3 Midfield                             1.41       3.65
                                            5 Right Floor                          2.24       5.89
                                            4 Right Peak                           2.24       8.12
                                            1 Left Floor                           3.61      11.73
                      5      11.89          1 Left Floor
                                            2 Left Peak                            2.24       2.24
                                            4 Right Peak                              2       4.24
                                            3 Midfield                             1.41       5.65
                                            5 Right Floor                          2.24       7.89
                                            1 Left Floor                              4      11.89
2 -> 3                1       11.3          2 Left Peak
                                            3 Midfield                             1.41       1.41
                                            4 Right Peak                           1.41       2.83
                                            5 Right Floor                          2.24       5.06
                                            1 Left Floor                              4       9.06
                                            2 Left Peak                            2.24       11.3
                      2      11.73          2 Left Peak
                                            3 Midfield                             1.41       1.41
                                            5 Right Floor                          2.24       3.65
                                            4 Right Peak                           2.24       5.89
                                            1 Left Floor                           3.61       9.49
                                            2 Left Peak                            2.24      11.73
                      3       13.1          2 Left Peak
                                            3 Midfield                             1.41       1.41
                                            1 Left Floor                           2.24       3.65
                                            4 Right Peak                           3.61       7.26
                                            5 Right Floor                          2.24       9.49
                                            2 Left Peak                            3.61       13.1
                      4      13.26          2 Left Peak
                                            3 Midfield                             1.41       1.41
                                            5 Right Floor                          2.24       3.65
                                            1 Left Floor                              4       7.65
                                            4 Right Peak                           3.61      11.26
                                            2 Left Peak                               2      13.26
                      5      14.04          2 Left Peak
                                            3 Midfield                             1.41       1.41
                                            4 Right Peak                           1.41       2.83
                                            1 Left Floor                           3.61       6.43
                                            5 Right Floor                             4      10.43
                                            2 Left Peak                            3.61      14.04
2 -> 4                1      10.94          2 Left Peak
                                            4 Right Peak                              2          2
                                            5 Right Floor                          2.24       4.24
                                            3 Midfield                             2.24       6.47
                                            1 Left Floor                           2.24       8.71
                                            2 Left Peak                            2.24      10.94
                      2      11.89          2 Left Peak
                                            4 Right Peak                              2          2
                                            3 Midfield                             1.41       3.41
                                            5 Right Floor                          2.24       5.65
                                            1 Left Floor                              4       9.65
                                            2 Left Peak                            2.24      11.89
                      3      11.89          2 Left Peak
                                            4 Right Peak                              2          2
                                            5 Right Floor                          2.24       4.24
                                            1 Left Floor                              4       8.24
                                            3 Midfield                             2.24      10.47
                                            2 Left Peak                            1.41      11.89
                      4      13.26          2 Left Peak
                                            4 Right Peak                              2          2
                                            3 Midfield                             1.41       3.41
                                            1 Left Floor                           2.24       5.65
                                            5 Right Floor                             4       9.65
                                            2 Left Peak                            3.61      13.26
                      5      13.68          2 Left Peak
                                            4 Right Peak                              2          2
                                            1 Left Floor                           3.61       5.61
                                            3 Midfield                             2.24       7.84
                                            5 Right Floor                          2.24      10.08
                                            2 Left Peak                            3.61      13.68
3 -> 4                1       11.3          3 Midfield
                                            4 Right Peak                           1.41       1.41
                                            5 Right Floor                          2.24       3.65
                                            1 Left Floor                              4       7.65
                                            2 Left Peak                            2.24       9.89
                                            3 Midfield                             1.41       11.3
                      2      11.73          3 Midfield
                                            4 Right Peak                           1.41       1.41
                                            5 Right Floor                          2.24       3.65
                                            2 Left Peak                            3.61       7.26
                                            1 Left Floor                           2.24       9.49
                                            3 Midfield                             2.24      11.73
                      3      11.89          3 Midfield
                                            4 Right Peak                           1.41       1.41
                                            2 Left Peak                               2       3.41
                                            1 Left Floor                           2.24       5.65
                                            5 Right Floor                             4       9.65
                                            3 Midfield                             2.24      11.89
                      4       13.1          3 Midfield
                                            4 Right Peak                           1.41       1.41
                                            1 Left Floor                           3.61       5.02
                                            2 Left Peak                            2.24       7.26
                                            5 Right Floor                          3.61      10.86
                                            3 Midfield                             2.24       13.1
                      5      13.26          3 Midfield
                                            4 Right Peak                           1.41       1.41
                                            2 Left Peak                               2       3.41
                                            5 Right Floor                          3.61       7.02
                                            1 Left Floor                              4      11.02
                                            3 Midfield                             2.24      13.26
3 -> 5                1      10.94          3 Midfield
                                            5 Right Floor                          2.24       2.24
                                            4 Right Peak                           2.24       4.47
                                            2 Left Peak                               2       6.47
                                            1 Left Floor                           2.24       8.71
                                            3 Midfield                             2.24      10.94
                      2      11.73          3 Midfield
                                            5 Right Floor                          2.24       2.24
                                            4 Right Peak                           2.24       4.47
                                            1 Left Floor                           3.61       8.08
                                            2 Left Peak                            2.24      10.31
                                            3 Midfield                             1.41      11.73
                      3      11.89          3 Midfield
                                            5 Right Floor                          2.24       2.24
                                            1 Left Floor                              4       6.24
                                            2 Left Peak                            2.24       8.47
                                            4 Right Peak                              2      10.47
                                            3 Midfield                             1.41      11.89
                      4       13.1          3 Midfield
                                            5 Right Floor                          2.24       2.24
                                            2 Left Peak                            3.61       5.84
                                            1 Left Floor                           2.24       8.08
                                            4 Right Peak                           3.61      11.68
                                            3 Midfield                             1.41       13.1
                      5      13.68          3 Midfield
                                            5 Right Floor                          2.24       2.24
                                            2 Left Peak                            3.61       5.84
                                            4 Right Peak                              2       7.84
                                            1 Left Floor                           3.61      11.45
                                            3 Midfield                             2.24      13.68

150 rows selected.

Elapsed: 00:00:00.14
Longest Routes
==============
ROOT_LEG       PATH_RNK   TOT_DIST    TOWN_ID NAME                             LEG_DIST   CUM_DIST
------------ ---------- ---------- ---------- ------------------------------ ---------- ----------
1 -> 2                1       13.1          1 Left Floor
                                            2 Left Peak                            2.24       2.24
                                            5 Right Floor                          3.61       5.84
                                            3 Midfield                             2.24       8.08
                                            4 Right Peak                           1.41       9.49
                                            1 Left Floor                           3.61       13.1
                      2      11.89          1 Left Floor
                                            2 Left Peak                            2.24       2.24
                                            4 Right Peak                              2       4.24
                                            3 Midfield                             1.41       5.65
                                            5 Right Floor                          2.24       7.89
                                            1 Left Floor                              4      11.89
                      3      11.73          1 Left Floor
                                            2 Left Peak                            2.24       2.24
                                            5 Right Floor                          3.61       5.84
                                            4 Right Peak                           2.24       8.08
                                            3 Midfield                             1.41       9.49
                                            1 Left Floor                           2.24      11.73
                      4      11.73          1 Left Floor
                                            2 Left Peak                            2.24       2.24
                                            3 Midfield                             1.41       3.65
                                            5 Right Floor                          2.24       5.89
                                            4 Right Peak                           2.24       8.12
                                            1 Left Floor                           3.61      11.73
                      5      10.94          1 Left Floor
                                            2 Left Peak                            2.24       2.24
                                            4 Right Peak                              2       4.24
                                            5 Right Floor                          2.24       6.47
                                            3 Midfield                             2.24       8.71
                                            1 Left Floor                           2.24      10.94
1 -> 4                1      13.68          1 Left Floor
                                            4 Right Peak                           3.61       3.61
                                            2 Left Peak                               2       5.61
                                            5 Right Floor                          3.61       9.21
                                            3 Midfield                             2.24      11.45
                                            1 Left Floor                           2.24      13.68
                      2      13.26          1 Left Floor
                                            4 Right Peak                           3.61       3.61
                                            2 Left Peak                               2       5.61
                                            3 Midfield                             1.41       7.02
                                            5 Right Floor                          2.24       9.26
                                            1 Left Floor                              4      13.26
                      3       13.1          1 Left Floor
                                            4 Right Peak                           3.61       3.61
                                            5 Right Floor                          2.24       5.84
                                            2 Left Peak                            3.61       9.45
                                            3 Midfield                             1.41      10.86
                                            1 Left Floor                           2.24       13.1
                      4       13.1          1 Left Floor
                                            4 Right Peak                           3.61       3.61
                                            3 Midfield                             1.41       5.02
                                            5 Right Floor                          2.24       7.26
                                            2 Left Peak                            3.61      10.86
                                            1 Left Floor                           2.24       13.1
                      5      11.73          1 Left Floor
                                            4 Right Peak                           3.61       3.61
                                            5 Right Floor                          2.24       5.84
                                            3 Midfield                             2.24       8.08
                                            2 Left Peak                            1.41       9.49
                                            1 Left Floor                           2.24      11.73
1 -> 5                1      14.04          1 Left Floor
                                            5 Right Floor                             4          4
                                            2 Left Peak                            3.61       7.61
                                            3 Midfield                             1.41       9.02
                                            4 Right Peak                           1.41      10.43
                                            1 Left Floor                           3.61      14.04
                      2      13.26          1 Left Floor
                                            5 Right Floor                             4          4
                                            2 Left Peak                            3.61       7.61
                                            4 Right Peak                              2       9.61
                                            3 Midfield                             1.41      11.02
                                            1 Left Floor                           2.24      13.26
                      3      13.26          1 Left Floor
                                            5 Right Floor                             4          4
                                            3 Midfield                             2.24       6.24
                                            2 Left Peak                            1.41       7.65
                                            4 Right Peak                              2       9.65
                                            1 Left Floor                           3.61      13.26
                      4      11.89          1 Left Floor
                                            5 Right Floor                             4          4
                                            4 Right Peak                           2.24       6.24
                                            2 Left Peak                               2       8.24
                                            3 Midfield                             1.41       9.65
                                            1 Left Floor                           2.24      11.89
                      5      11.89          1 Left Floor
                                            5 Right Floor                             4          4
                                            3 Midfield                             2.24       6.24
                                            4 Right Peak                           1.41       7.65
                                            2 Left Peak                               2       9.65
                                            1 Left Floor                           2.24      11.89
2 -> 5                1      14.04          2 Left Peak
                                            5 Right Floor                          3.61       3.61
                                            1 Left Floor                              4       7.61
                                            4 Right Peak                           3.61      11.21
                                            3 Midfield                             1.41      12.63
                                            2 Left Peak                            1.41      14.04
                      2      13.68          2 Left Peak
                                            5 Right Floor                          3.61       3.61
                                            3 Midfield                             2.24       5.84
                                            1 Left Floor                           2.24       8.08
                                            4 Right Peak                           3.61      11.68
                                            2 Left Peak                               2      13.68
                      3      13.26          2 Left Peak
                                            5 Right Floor                          3.61       3.61
                                            1 Left Floor                              4       7.61
                                            3 Midfield                             2.24       9.84
                                            4 Right Peak                           1.41      11.26
                                            2 Left Peak                               2      13.26
                      4       13.1          2 Left Peak
                                            5 Right Floor                          3.61       3.61
                                            4 Right Peak                           2.24       5.84
                                            1 Left Floor                           3.61       9.45
                                            3 Midfield                             2.24      11.68
                                            2 Left Peak                            1.41       13.1
                      5      11.73          2 Left Peak
                                            5 Right Floor                          3.61       3.61
                                            4 Right Peak                           2.24       5.84
                                            3 Midfield                             1.41       7.26
                                            1 Left Floor                           2.24       9.49
                                            2 Left Peak                            2.24      11.73
3 -> 5                1      13.68          3 Midfield
                                            5 Right Floor                          2.24       2.24
                                            2 Left Peak                            3.61       5.84
                                            4 Right Peak                              2       7.84
                                            1 Left Floor                           3.61      11.45
                                            3 Midfield                             2.24      13.68
                      2      13.26          3 Midfield
                                            5 Right Floor                          2.24       2.24
                                            1 Left Floor                              4       6.24
                                            4 Right Peak                           3.61       9.84
                                            2 Left Peak                               2      11.84
                                            3 Midfield                             1.41      13.26
                      3       13.1          3 Midfield
                                            5 Right Floor                          2.24       2.24
                                            2 Left Peak                            3.61       5.84
                                            1 Left Floor                           2.24       8.08
                                            4 Right Peak                           3.61      11.68
                                            3 Midfield                             1.41       13.1
                      4      11.89          3 Midfield
                                            5 Right Floor                          2.24       2.24
                                            1 Left Floor                              4       6.24
                                            2 Left Peak                            2.24       8.47
                                            4 Right Peak                              2      10.47
                                            3 Midfield                             1.41      11.89
                      5      11.73          3 Midfield
                                            5 Right Floor                          2.24       2.24
                                            4 Right Peak                           2.24       4.47
                                            1 Left Floor                           3.61       8.08
                                            2 Left Peak                            2.24      10.31
                                            3 Midfield                             1.41      11.73

150 rows selected.

Elapsed: 00:00:00.16

Summary of Results
Here we list the best solutions found for each root leg. * denotes optimum.

Shortest Routes
===============
ROOT_LEG       PATH_RNK   TOT_DIST
------------ ---------- ----------
1 -> 2                1      10.94*
2 -> 3                1       11.3
2 -> 4                1      10.94*
3 -> 4                1       11.3
3 -> 5                1      10.94*

Longest Routes
==============
ROOT_LEG       PATH_RNK   TOT_DIST
------------ ---------- ----------
1 -> 2                1       13.1
1 -> 4                1      13.68
1 -> 5                1      14.04*
2 -> 5                1      14.04*
3 -> 5                1      13.68

Test Problem 2: USCA312

I used keep values of 2 and 2 and solved the problems in around 85 seconds.

Shortest Routes
===============
ROOT_LEG       PATH_RNK   TOT_DIST    TOWN_ID NAME                             LEG_DIST   CUM_DIST
------------ ---------- ---------- ---------- ------------------------------ ---------- ----------
184 -> 208            1   55744.86        184 Norfolk, VA
                                          208 Portsmouth, VA                       1.19       1.19
                                          294 Washington, DC                     151.18     152.37
                                           18 Baltimore, MD                       40.06     192.43
                                          141 Lancaster, PA                       55.83     248.26
                                          129 Johnstown, PA                       16.64      264.9
                                          217 Reading, PA                          27.2      292.1
                                          302 Wilmington, DE                      48.48     340.58
                                          198 Philadelphia, PA                    30.05     370.63
                                          283 Trenton, NJ                         34.35     404.99
                                           83 Elizabeth, NJ                       48.02     453.01
                                          177 Newark, NJ                           5.61     458.62
                                          128 Jersey City, NJ                      6.57     465.19
                                          181 New York, NY                         5.04     470.23
                                          194 Paterson, NJ                        18.08     488.32
                                          299 White Plains, NY                    29.39     517.71
                                          266 Stamford, CT                        15.36     533.07
                                           39 Bridgeport, CT                      24.55     557.61
                                          179 New Haven, CT                       21.46     579.08
                                          163 Meriden, CT                         17.96     597.04
                                          178 New Britain,CT                       8.71     605.75
                                          117 Hartford, CT                         9.62     615.37
                                          263 Springfield, MA                     24.25     639.62
                                          202 Pittsfield, MA                      51.31     690.93
                                          285 Troy, NY                            36.34     727.27
                                            3 Albany, NY                           6.88     734.16
                                          251 Schenectady, NY                     16.88     751.04
                                          289 Utica, NY                           91.52     842.56
                                          272 Syracuse, NY                        63.31     905.86
                                           29 Binghamtom, NY                      67.49     973.35
                                          301 Wilkes-Barre, PA                    58.97    1032.33
                                            6 Allentown, PA                       51.68       1084
                                          116 Harrisburg, PA                      99.07    1183.07
                                           13 Atlantic City, NJ                  181.31    1364.38
                                           50 Central Islip, NY                  129.74    1494.12
                                          210 Providence, RI                     142.75    1636.87
                                           91 Fall River, MA                      19.72    1656.59
                                           40 Brockton, MA                        28.03    1684.61
                                           34 Boston, MA                          19.21    1703.83
                                           46 Cambridge, MA                        3.36    1707.19
                                          155 Lowell, MA                          23.03    1730.22
                                          145 Lawrence, MA                        11.73    1741.95
                                          159 Manchester, NH                      28.36     1770.3
                                           67 Concord, NH                         15.76    1786.06
                                          307 Worcester, MA                       67.84     1853.9
                                           38 Brattleboro, VT                     66.17    1920.07
                                          171 Montpelier, VT                      97.37    2017.43
                                           43 Burlington, VT                      46.45    2063.89
                                          172 Montreal, QC                        97.19    2161.08
                                          284 Trois-Rivieres, QC                  98.03     2259.1
                                          254 Sherbrooke, QC                      93.08    2352.18
                                          205 Portland, ME                       157.79    2509.97
                                          207 Portsmouth, NH                      53.74    2563.71
                                           15 Augusta, ME                        109.27    2672.99
                                           19 Bangor, ME                          77.07    2750.06
                                           98 Fredericton, NB                    171.89    2921.95
                                          229 Saint John, NB                      58.53    2980.48
                                          169 Moncton, NB                         99.94    3080.42
                                          113 Halifax, NS                        117.55    3197.97
                                           55 Charlottetown, PE                  100.98    3298.95
                                          271 Syndey, NS                         212.55     3511.5
                                          230 Saint John's, NF                   514.07    4025.57
                                          213 Quebec City, QC                   1290.47    5316.03
                                          191 Ottawa, ON                         400.01    5716.04
                                          137 Kingston, ON                        85.39    5801.42
                                           24 Belleville, ON                      49.28    5850.71
                                          197 Peterborough, ON                    72.87    5923.57
                                          282 Toronto, ON                         82.62    6006.19
                                           42 Burlington, ONT                     33.92    6040.11
                                          115 Hamilton, ON                        19.51    6059.62
                                           37 Brantford, ON                       28.55    6088.16
                                          111 Guelph, ON                           29.4    6117.56
                                          138 Kitchener, ON                       10.29    6127.86
                                          152 London, ON                          67.63    6195.49
                                           86 Erie, PA                            97.44    6292.93
                                          310 Youngstown, OH                      81.12    6374.05
                                          267 Steubenville, OH                    50.45     6424.5
                                          297 Wheeling, WV                        21.97    6446.47
                                          201 Pittsburgh, PA                      56.46    6502.93
                                           47 Canton, OH                          98.69    6601.61
                                            2 Akron, OH                            21.8    6623.41
                                           61 Cleveland, OH                       31.35    6654.76
                                          304 Windsor, ON                        102.38    6757.14
                                           76 Detroit, MI                          7.21    6764.35
                                            9 Ann Arbor, MI                       47.21    6811.56
                                          125 Jackson, MI                         46.67    6858.23
                                          142 Lansing, MI                         35.28    6893.51
                                           21 Battle Creek, MI                    51.65    6945.16
                                          132 Kalamazoo, MI                       28.23    6973.39
                                          259 South Bend, IN                      62.16    7035.55
                                          104 Gary, IN                            76.01    7111.56
                                           58 Chicago, IL                         27.47    7139.03
                                          135 Kenosha, WI                         52.13    7191.16
                                          214 Racine, WI                          10.12    7201.28
                                          165 Milwaukee, WI                       23.24    7224.52
                                          224 Rockford, IL                        97.71    7322.22
                                          158 Madison, WI                         59.34    7381.56
                                           78 Dubuque, IA                         95.84     7477.4
                                           49 Cedar Rapids, IA                    75.74    7553.14
                                          124 Iowa City, IA                       25.25    7578.39
                                          295 Waterloo, IA                        80.35    7658.74
                                          222 Rochester, MN                         106    7764.75
                                          233 Saint Paul, MN                      76.94    7841.69
                                          166 Minneapolis, MN                     12.03    7853.72
                                          228 Saint Cloud, MN                     73.94    7927.66
                                           79 Duluth, MN                         165.27    8092.93
                                          270 Superior, WI                         4.32    8097.25
                                           81 Eau Claire, WI                     138.42    8235.67
                                          108 Green Bay, WI                      241.21    8476.87
                                          253 Sheboygan, WI                       57.13       8534
                                          106 Grand Rapids, MI                   151.51     8685.5
                                          226 Saginaw, MI                        122.77    8808.27
                                           22 Bay City, MI                        12.83     8821.1
                                           94 Flint, MI                           42.55    8863.65
                                          280 Toledo, OH                          93.64    8957.28
                                          149 Lima, OH                            74.14    9031.42
                                          265 Springfield, OH                     60.13    9091.56
                                           71 Dayton, OH                          28.81    9120.37
                                          114 Hamilton, OH                        35.63       9156
                                           60 Cincinnati, OH                      17.93    9173.93
                                          147 Lexington, KY                       81.08    9255.01
                                          154 Louisville, KY                      90.44    9345.45
                                           35 Bowling Green, KY                    99.3    9444.76
                                          175 Nashville, TN                       61.65     9506.4
                                          122 Huntsville, AL                     100.13    9606.53
                                          100 Gadsden, AL                         63.64    9670.18
                                           30 Birmingham, AL                       64.7    9734.88
                                          170 Montgomery, AL                      86.96    9821.85
                                           65 Columbus, GA                         90.9    9912.75
                                           12 Atlanta, GA                         98.17   10010.92
                                          157 Macon, GA                           81.64   10092.56
                                           14 Augusta, GA                        122.52   10215.08
                                          110 Greenville, SC                      99.77   10314.85
                                          260 Spartanburg, NC                     32.62   10347.46
                                           54 Charlotte, NC                       77.64   10425.11
                                          306 Winston-Salem, NC                   73.13   10498.24
                                          109 Greensboro, NC                       31.3   10529.54
                                          215 Raleigh, NC                         82.35   10611.89
                                           80 Durham, NC                          23.61    10635.5
                                          221 Roanoke, VA                        113.91   10749.41
                                           53 Charleston, WV                     138.61   10888.02
                                           11 Ashland, KY                         70.03   10958.04
                                          312 Zanesville, OH                     109.84   11067.89
                                           66 Columbus, OH                        68.11   11135.99
                                          174 Muncie, IN                         165.75   11301.74
                                          123 Indianapolis, IN                    60.87   11362.61
                                          276 Terre Haute, IN                     89.24   11451.85
                                          288 Urbana, IL                           70.6   11522.45
                                           51 Champaign, IL                        2.52   11524.98
                                           73 Decatur, IL                         52.73   11577.71
                                           32 Bloomington, IL                     44.57   11622.28
                                          196 Peoria, IL                           43.6   11665.88
                                          262 Springfield, IL                     61.75   11727.63
                                          232 Saint Louis, MO                     89.73   11817.36
                                           63 Columbia, MO                       149.28   11966.64
                                          264 Springfield, MO                    137.23   12103.88
                                          130 Joplin, MO                          84.44   12188.32
                                          287 Tulsa, OK                          120.76   12309.07
                                          188 Oklahoma City, OK                  115.46   12424.54
                                           85 Enid, OK                            68.83   12493.37
                                          300 Wichita, KS                         97.07   12590.44
                                          236 Salina, KS                          81.55   12671.99
                                          281 Topeka, KS                         134.35   12806.34
                                          133 Kansas City, KS                     72.73   12879.07
                                          134 Kansas City, MO                      3.52    12882.6
                                          231 Saint Joseph, MO                    49.79   12932.39
                                          150 Lincoln, NE                        144.56   13076.95
                                          189 Omaha, NE                           59.52   13136.47
                                          257 Sioux City, IA                      91.54   13228.01
                                          258 Sioux Falls, SD                     75.45   13303.46
                                           92 Fargo, ND                          229.97   13533.43
                                          305 Winnipeg, MB                       211.94   13745.38
                                           36 Brandon, MB                        187.08   13932.46
                                          167 Minot, ND                          148.55      14081
                                           31 Bismarck, ND                       104.59   14185.59
                                          200 Pierre, SD                         171.22   14356.81
                                          216 Rapid City, SD                     199.98   14556.79
                                           57 Cheyenne, WY                       230.96   14787.75
                                           74 Denver, CO                          97.46   14885.21
                                           62 Colorado Springs, CO                63.56   14948.77
                                          212 Pueblo, CO                          42.63    14991.4
                                          246 Santa Fe, NM                       199.75   15191.16
                                            4 Albuquerque, NM                     64.52   15255.68
                                          102 Gallup, NM                         147.72    15403.4
                                           93 Flagstaff, AZ                      202.26   15605.66
                                          199 Phoenix, AZ                        124.38   15730.04
                                          286 Tucson, AZ                         116.06    15846.1
                                          311 Yuma, AZ                           257.86   16103.95
                                          240 San Diego, CA                      175.01   16278.97
                                          239 San Bernardino, CA                  96.69   16375.66
                                          193 Pasadena, CA                        59.12   16434.78
                                          153 Los Angeles, CA                      9.51   16444.29
                                          244 Santa Barbara, CA                  103.68   16547.97
                                           17 Bakersfield, CA                     80.84   16628.81
                                           99 Fresno, CA                         108.31   16737.12
                                          242 San Jose, CA                       152.25   16889.37
                                          245 Santa Cruz, CA                      26.93    16916.3
                                          186 Oakland, CA                         59.72   16976.02
                                          241 San Francisco, CA                   10.47   16986.49
                                           26 Berkeley, CA                        12.14   16998.63
                                          268 Stockton, CA                        68.11   17066.74
                                          225 Sacramento, CA                      45.35   17112.09
                                           48 Carson City, NV                    125.92      17238
                                          219 Reno, NV                            25.48   17263.49
                                           88 Eureka, CA                         313.15   17576.64
                                           87 Eugene, OR                         236.57    17813.2
                                          235 Salem, OR                           61.63   17874.84
                                          206 Portland, OR                        47.17   17922.01
                                          273 Tacoma, WA                         120.57   18042.58
                                          252 Seattle, WA                         25.62    18068.2
                                           25 Bellingham, WA                      80.42   18148.62
                                          290 Vancouver, BC                       56.66   18205.28
                                          291 Victoria, BC                        41.45   18246.73
                                          308 Yakima, WA                         246.37    18493.1
                                          293 Walla Walla, WA                    153.98   18647.07
                                          261 Spokane, WA                        127.07   18774.15
                                           33 Boise, ID                          291.99   19066.14
                                          203 Pocatello, ID                      264.67   19330.81
                                          187 Ogden, UT                          118.47   19449.28
                                          237 Salt Lake City, UT                  32.44   19481.72
                                          211 Provo, UT                           39.79   19521.51
                                          105 Grand Junction, CO                 229.45   19750.96
                                          255 Sheridan, WY                       411.17   20162.14
                                           27 Billings, MT                       126.61   20288.75
                                          107 Great Falls, MT                    226.94   20515.69
                                          118 Helena, MT                           80.7   20596.39
                                           44 Butte, MT                           53.31    20649.7
                                          146 Lethbridge, AB                     251.24   20900.94
                                           45 Calgary, AB                        132.75   21033.69
                                           82 Edmonton, AB                       173.35   21207.04
                                          161 Medicine Hat, AB                   312.59   21519.63
                                          248 Saskatoon, SK                      315.13   21834.76
                                          173 Moose Jaw, SK                      149.68   21984.44
                                          218 Regina, SK                          62.04   22046.48
                                           77 Dodge City, KS                     933.12    22979.6
                                            7 Amarillo, TX                       215.16   23194.75
                                          156 Lubbock, TX                        113.61   23308.37
                                            1 Abilene, TX                        166.25   23474.62
                                           97 Ft Worth, TX                       167.57   23642.19
                                           69 Dallas, TX                          36.19   23678.39
                                          292 Waco, TX                            88.57   23766.96
                                           16 Austin, TX                          97.72   23864.68
                                          238 San Antonio, TX                     77.99   23942.67
                                          143 Laredo, TX                         149.89   24092.56
                                           68 Corpus Christi, TX                 147.28   24239.84
                                          121 Houston, TX                        195.26    24435.1
                                          103 Galveston, TX                       50.45   24485.55
                                          204 Port Arthur, TX                     72.86   24558.41
                                           23 Beaumont, TX                        17.62   24576.03
                                          256 Shreveport, LA                     170.28   24746.32
                                          160 Marshall, TX                        42.67   24788.99
                                          277 Texarkana, TX                       64.71    24853.7
                                           95 Ft Smith, AR                       137.64   24991.34
                                          151 Little Rock, AR                    152.27   25143.61
                                          162 Memphis, TN                         157.3   25300.91
                                          192 Paducah, KY                        166.97   25467.88
                                           89 Evansville, IN                      94.86   25562.74
                                          140 Lafayette, IN                      175.16    25737.9
                                           96 Ft Wayne, IN                       130.36   25868.26
                                          139 Knoxville, TN                      366.84    26235.1
                                           10 Asheville, NC                       97.65   26332.75
                                           64 Columbia, SC                       152.46   26485.21
                                           52 Charleston, SC                      113.9   26599.11
                                          250 Savannah, GA                         93.9   26693.01
                                          127 Jacksonville, FL                   126.96   26819.97
                                          101 Gainesville, FL                     65.95   26885.92
                                           72 Daytona Beach, FL                   94.98    26980.9
                                          190 Orlando, FL                         52.58   27033.48
                                          275 Tampa, FL                           85.01   27118.49
                                          234 Saint Petersburg, FL                19.54   27138.03
                                          247 Sarasota, FL                        31.73   27169.75
                                          296 West Palm Beach, FL                176.46   27346.21
                                          164 Miami, FL                           65.75   27411.96
                                          136 Key West, FL                       138.36   27550.32
                                          274 Tallahassee, FL                    441.61   27991.93
                                          195 Pensacola, FL                      202.87    28194.8
                                          168 Mobile, AL                          60.12   28254.92
                                           28 Biloxi, MS                          61.74   28316.66
                                          112 Gulfport, MS                        14.47   28331.13
                                          180 New Orleans, LA                     73.62   28404.74
                                           20 Baton Rouge, LA                     82.09   28486.83
                                          176 Natchez, MS                         78.58   28565.41
                                          126 Jackson, MS                         98.43   28663.84
                                           56 Chattanooga, TN                    386.64   29050.48
                                          303 Wilmington, NC                     512.01   29562.49
                                          220 Richmond, VA                       232.37   29794.87
                                           41 Buffalo, NY                        381.27   30176.14
                                          182 Niagara Falls, ON                   20.54   30196.68
                                          227 Saint Catherines, ON                 8.92   30205.61
                                          223 Rochester, NY                      107.53   30313.14
                                          185 North Bay, ON                      260.14   30573.28
                                          269 Sudbury, ON                            93   30666.28
                                          279 Timmins, ON                        140.17   30806.45
                                          249 Sault Ste Marie, ON                260.02   31066.47
                                          278 Thunder Bay, ON                    357.29   31423.76
                                           75 Des Moines, IA                     553.68   31977.44
                                           84 El Paso, TX                       1119.89   33097.34
                                          144 Las Vegas, NV                      671.08   33768.41
                                          209 Prince Rupert, BC                 1638.53   35406.94
                                          131 Juneau, AK                         390.43   35797.37
                                          298 Whitehorse, YK                     172.32   35969.69
                                           70 Dawson, YT                         362.92   36332.61
                                           90 Fairbanks, AK                         596   36928.62
                                            8 Anchorage, AK                      292.11   37220.72
                                          183 Nome, AK                          1095.15   38315.87
                                          148 Lihue, HI                          2967.4   41283.27
                                          120 Honolulu, HI                        114.5   41397.77
                                          119 Hilo, HI                              176   41573.77
                                          309 Yellowknife, NT                   4111.93    45685.7
                                           59 Churchill, MB                     1431.71   47117.42
                                            5 Alert, NT                         2742.38    49859.8
                                          243 San Juan, PR                      4433.43   54293.23
                                          184 Norfolk, VA                       1451.64   55744.86
                      2   55746.24        184 Norfolk, VA
                                          208 Portsmouth, VA                       1.19       1.19
                                          294 Washington, DC                     151.18     152.37
                                           18 Baltimore, MD                       40.06     192.43
                                          141 Lancaster, PA                       55.83     248.26
                                          129 Johnstown, PA                       16.64      264.9
                                          217 Reading, PA                          27.2      292.1
                                          302 Wilmington, DE                      48.48     340.58
                                          198 Philadelphia, PA                    30.05     370.63
                                          283 Trenton, NJ                         34.35     404.99
                                           83 Elizabeth, NJ                       48.02     453.01
                                          177 Newark, NJ                           5.61     458.62
                                          128 Jersey City, NJ                      6.57     465.19
                                          181 New York, NY                         5.04     470.23
                                          194 Paterson, NJ                        18.08     488.32
                                          299 White Plains, NY                    29.39     517.71
                                          266 Stamford, CT                        15.36     533.07
                                           39 Bridgeport, CT                      24.55     557.61
                                          179 New Haven, CT                       21.46     579.08
                                          163 Meriden, CT                         17.96     597.04
                                          178 New Britain,CT                       8.71     605.75
                                          117 Hartford, CT                         9.62     615.37
                                          263 Springfield, MA                     24.25     639.62
                                          202 Pittsfield, MA                      51.31     690.93
                                          285 Troy, NY                            36.34     727.27
                                            3 Albany, NY                           6.88     734.16
                                          251 Schenectady, NY                     16.88     751.04
                                          289 Utica, NY                           91.52     842.56
                                          272 Syracuse, NY                        63.31     905.86
                                           29 Binghamtom, NY                      67.49     973.35
                                          301 Wilkes-Barre, PA                    58.97    1032.33
                                            6 Allentown, PA                       51.68       1084
                                          116 Harrisburg, PA                      99.07    1183.07
                                           13 Atlantic City, NJ                  181.31    1364.38
                                           50 Central Islip, NY                  129.74    1494.12
                                          210 Providence, RI                     142.75    1636.87
                                           91 Fall River, MA                      19.72    1656.59
                                           40 Brockton, MA                        28.03    1684.61
                                           34 Boston, MA                          19.21    1703.83
                                           46 Cambridge, MA                        3.36    1707.19
                                          155 Lowell, MA                          23.03    1730.22
                                          145 Lawrence, MA                        11.73    1741.95
                                          159 Manchester, NH                      28.36     1770.3
                                           67 Concord, NH                         15.76    1786.06
                                          307 Worcester, MA                       67.84     1853.9
                                           38 Brattleboro, VT                     66.17    1920.07
                                          171 Montpelier, VT                      97.37    2017.43
                                           43 Burlington, VT                      46.45    2063.89
                                          172 Montreal, QC                        97.19    2161.08
                                          284 Trois-Rivieres, QC                  98.03     2259.1
                                          254 Sherbrooke, QC                      93.08    2352.18
                                          205 Portland, ME                       157.79    2509.97
                                          207 Portsmouth, NH                      53.74    2563.71
                                           15 Augusta, ME                        109.27    2672.99
                                           19 Bangor, ME                          77.07    2750.06
                                           98 Fredericton, NB                    171.89    2921.95
                                          229 Saint John, NB                      58.53    2980.48
                                          169 Moncton, NB                         99.94    3080.42
                                          113 Halifax, NS                        117.55    3197.97
                                           55 Charlottetown, PE                  100.98    3298.95
                                          271 Syndey, NS                         212.55     3511.5
                                          230 Saint John's, NF                   514.07    4025.57
                                          213 Quebec City, QC                   1290.47    5316.03
                                          191 Ottawa, ON                         400.01    5716.04
                                          137 Kingston, ON                        85.39    5801.42
                                           24 Belleville, ON                      49.28    5850.71
                                          197 Peterborough, ON                    72.87    5923.57
                                          282 Toronto, ON                         82.62    6006.19
                                           42 Burlington, ONT                     33.92    6040.11
                                          115 Hamilton, ON                        19.51    6059.62
                                           37 Brantford, ON                       28.55    6088.16
                                          111 Guelph, ON                           29.4    6117.56
                                          138 Kitchener, ON                       10.29    6127.86
                                          152 London, ON                          67.63    6195.49
                                           86 Erie, PA                            97.44    6292.93
                                          310 Youngstown, OH                      81.12    6374.05
                                          267 Steubenville, OH                    50.45     6424.5
                                          297 Wheeling, WV                        21.97    6446.47
                                          201 Pittsburgh, PA                      56.46    6502.93
                                           47 Canton, OH                          98.69    6601.61
                                            2 Akron, OH                            21.8    6623.41
                                           61 Cleveland, OH                       31.35    6654.76
                                          304 Windsor, ON                        102.38    6757.14
                                           76 Detroit, MI                          7.21    6764.35
                                            9 Ann Arbor, MI                       47.21    6811.56
                                          125 Jackson, MI                         46.67    6858.23
                                          142 Lansing, MI                         35.28    6893.51
                                           21 Battle Creek, MI                    51.65    6945.16
                                          132 Kalamazoo, MI                       28.23    6973.39
                                          259 South Bend, IN                      62.16    7035.55
                                          104 Gary, IN                            76.01    7111.56
                                           58 Chicago, IL                         27.47    7139.03
                                          135 Kenosha, WI                         52.13    7191.16
                                          214 Racine, WI                          10.12    7201.28
                                          165 Milwaukee, WI                       23.24    7224.52
                                          224 Rockford, IL                        97.71    7322.22
                                          158 Madison, WI                         59.34    7381.56
                                           78 Dubuque, IA                         95.84     7477.4
                                           49 Cedar Rapids, IA                    75.74    7553.14
                                          124 Iowa City, IA                       25.25    7578.39
                                          295 Waterloo, IA                        80.35    7658.74
                                          222 Rochester, MN                         106    7764.75
                                          233 Saint Paul, MN                      76.94    7841.69
                                          166 Minneapolis, MN                     12.03    7853.72
                                          228 Saint Cloud, MN                     73.94    7927.66
                                           79 Duluth, MN                         165.27    8092.93
                                          270 Superior, WI                         4.32    8097.25
                                           81 Eau Claire, WI                     138.42    8235.67
                                          108 Green Bay, WI                      241.21    8476.87
                                          253 Sheboygan, WI                       57.13       8534
                                          106 Grand Rapids, MI                   151.51     8685.5
                                          226 Saginaw, MI                        122.77    8808.27
                                           22 Bay City, MI                        12.83     8821.1
                                           94 Flint, MI                           42.55    8863.65
                                          280 Toledo, OH                          93.64    8957.28
                                          149 Lima, OH                            74.14    9031.42
                                          265 Springfield, OH                     60.13    9091.56
                                           71 Dayton, OH                          28.81    9120.37
                                          114 Hamilton, OH                        35.63       9156
                                           60 Cincinnati, OH                      17.93    9173.93
                                          147 Lexington, KY                       81.08    9255.01
                                          154 Louisville, KY                      90.44    9345.45
                                           35 Bowling Green, KY                    99.3    9444.76
                                          175 Nashville, TN                       61.65     9506.4
                                          122 Huntsville, AL                     100.13    9606.53
                                          100 Gadsden, AL                         63.64    9670.18
                                           30 Birmingham, AL                       64.7    9734.88
                                          170 Montgomery, AL                      86.96    9821.85
                                           65 Columbus, GA                         90.9    9912.75
                                           12 Atlanta, GA                         98.17   10010.92
                                          157 Macon, GA                           81.64   10092.56
                                           14 Augusta, GA                        122.52   10215.08
                                          110 Greenville, SC                      99.77   10314.85
                                          260 Spartanburg, NC                     32.62   10347.46
                                           54 Charlotte, NC                       77.64   10425.11
                                          306 Winston-Salem, NC                   73.13   10498.24
                                          109 Greensboro, NC                       31.3   10529.54
                                          215 Raleigh, NC                         82.35   10611.89
                                           80 Durham, NC                          23.61    10635.5
                                          221 Roanoke, VA                        113.91   10749.41
                                           53 Charleston, WV                     138.61   10888.02
                                           11 Ashland, KY                         70.03   10958.04
                                          312 Zanesville, OH                     109.84   11067.89
                                           66 Columbus, OH                        68.11   11135.99
                                          174 Muncie, IN                         165.75   11301.74
                                          123 Indianapolis, IN                    60.87   11362.61
                                          276 Terre Haute, IN                     89.24   11451.85
                                          288 Urbana, IL                           70.6   11522.45
                                           51 Champaign, IL                        2.52   11524.98
                                           73 Decatur, IL                         52.73   11577.71
                                           32 Bloomington, IL                     44.57   11622.28
                                          196 Peoria, IL                           43.6   11665.88
                                          262 Springfield, IL                     61.75   11727.63
                                          232 Saint Louis, MO                     89.73   11817.36
                                           63 Columbia, MO                       149.28   11966.64
                                          264 Springfield, MO                    137.23   12103.88
                                          130 Joplin, MO                          84.44   12188.32
                                          287 Tulsa, OK                          120.76   12309.07
                                          188 Oklahoma City, OK                  115.46   12424.54
                                           85 Enid, OK                            68.83   12493.37
                                          300 Wichita, KS                         97.07   12590.44
                                          236 Salina, KS                          81.55   12671.99
                                          281 Topeka, KS                         134.35   12806.34
                                          133 Kansas City, KS                     72.73   12879.07
                                          134 Kansas City, MO                      3.52    12882.6
                                          231 Saint Joseph, MO                    49.79   12932.39
                                          150 Lincoln, NE                        144.56   13076.95
                                          189 Omaha, NE                           59.52   13136.47
                                          257 Sioux City, IA                      91.54   13228.01
                                          258 Sioux Falls, SD                     75.45   13303.46
                                           92 Fargo, ND                          229.97   13533.43
                                          305 Winnipeg, MB                       211.94   13745.38
                                           36 Brandon, MB                        187.08   13932.46
                                          167 Minot, ND                          148.55      14081
                                           31 Bismarck, ND                       104.59   14185.59
                                          200 Pierre, SD                         171.22   14356.81
                                          216 Rapid City, SD                     199.98   14556.79
                                           57 Cheyenne, WY                       230.96   14787.75
                                           74 Denver, CO                          97.46   14885.21
                                           62 Colorado Springs, CO                63.56   14948.77
                                          212 Pueblo, CO                          42.63    14991.4
                                          246 Santa Fe, NM                       199.75   15191.16
                                            4 Albuquerque, NM                     64.52   15255.68
                                          102 Gallup, NM                         147.72    15403.4
                                           93 Flagstaff, AZ                      202.26   15605.66
                                          199 Phoenix, AZ                        124.38   15730.04
                                          286 Tucson, AZ                         116.06    15846.1
                                          311 Yuma, AZ                           257.86   16103.95
                                          240 San Diego, CA                      175.01   16278.97
                                          239 San Bernardino, CA                  96.69   16375.66
                                          193 Pasadena, CA                        59.12   16434.78
                                          153 Los Angeles, CA                      9.51   16444.29
                                          244 Santa Barbara, CA                  103.68   16547.97
                                           17 Bakersfield, CA                     80.84   16628.81
                                           99 Fresno, CA                         108.31   16737.12
                                          242 San Jose, CA                       152.25   16889.37
                                          245 Santa Cruz, CA                      26.93    16916.3
                                          186 Oakland, CA                         59.72   16976.02
                                          241 San Francisco, CA                   10.47   16986.49
                                           26 Berkeley, CA                        12.14   16998.63
                                          268 Stockton, CA                        68.11   17066.74
                                          225 Sacramento, CA                      45.35   17112.09
                                           48 Carson City, NV                    125.92      17238
                                          219 Reno, NV                            25.48   17263.49
                                           88 Eureka, CA                         313.15   17576.64
                                           87 Eugene, OR                         236.57    17813.2
                                          235 Salem, OR                           61.63   17874.84
                                          206 Portland, OR                        47.17   17922.01
                                          273 Tacoma, WA                         120.57   18042.58
                                          252 Seattle, WA                         25.62    18068.2
                                           25 Bellingham, WA                      80.42   18148.62
                                          290 Vancouver, BC                       56.66   18205.28
                                          291 Victoria, BC                        41.45   18246.73
                                          308 Yakima, WA                         246.37    18493.1
                                          293 Walla Walla, WA                    153.98   18647.07
                                          261 Spokane, WA                        127.07   18774.15
                                           33 Boise, ID                          291.99   19066.14
                                          203 Pocatello, ID                      264.67   19330.81
                                          187 Ogden, UT                          118.47   19449.28
                                          237 Salt Lake City, UT                  32.44   19481.72
                                          211 Provo, UT                           39.79   19521.51
                                          105 Grand Junction, CO                 229.45   19750.96
                                          255 Sheridan, WY                       411.17   20162.14
                                           27 Billings, MT                       126.61   20288.75
                                          107 Great Falls, MT                    226.94   20515.69
                                          118 Helena, MT                           80.7   20596.39
                                           44 Butte, MT                           53.31    20649.7
                                          146 Lethbridge, AB                     251.24   20900.94
                                           45 Calgary, AB                        132.75   21033.69
                                           82 Edmonton, AB                       173.35   21207.04
                                          161 Medicine Hat, AB                   312.59   21519.63
                                          248 Saskatoon, SK                      315.13   21834.76
                                          173 Moose Jaw, SK                      149.68   21984.44
                                          218 Regina, SK                          62.04   22046.48
                                           77 Dodge City, KS                     933.12    22979.6
                                            7 Amarillo, TX                       215.16   23194.75
                                          156 Lubbock, TX                        113.61   23308.37
                                            1 Abilene, TX                        166.25   23474.62
                                           97 Ft Worth, TX                       167.57   23642.19
                                           69 Dallas, TX                          36.19   23678.39
                                          292 Waco, TX                            88.57   23766.96
                                           16 Austin, TX                          97.72   23864.68
                                          238 San Antonio, TX                     77.99   23942.67
                                          143 Laredo, TX                         149.89   24092.56
                                           68 Corpus Christi, TX                 147.28   24239.84
                                          121 Houston, TX                        195.26    24435.1
                                          103 Galveston, TX                       50.45   24485.55
                                          204 Port Arthur, TX                     72.86   24558.41
                                           23 Beaumont, TX                        17.62   24576.03
                                          160 Marshall, TX                       170.89   24746.92
                                          256 Shreveport, LA                      42.67   24789.59
                                          277 Texarkana, TX                       65.49   24855.08
                                           95 Ft Smith, AR                       137.64   24992.72
                                          151 Little Rock, AR                    152.27      25145
                                          162 Memphis, TN                         157.3   25302.29
                                          192 Paducah, KY                        166.97   25469.26
                                           89 Evansville, IN                      94.86   25564.12
                                          140 Lafayette, IN                      175.16   25739.28
                                           96 Ft Wayne, IN                       130.36   25869.64
                                          139 Knoxville, TN                      366.84   26236.49
                                           10 Asheville, NC                       97.65   26334.13
                                           64 Columbia, SC                       152.46    26486.6
                                           52 Charleston, SC                      113.9   26600.49
                                          250 Savannah, GA                         93.9   26694.39
                                          127 Jacksonville, FL                   126.96   26821.35
                                          101 Gainesville, FL                     65.95    26887.3
                                           72 Daytona Beach, FL                   94.98   26982.28
                                          190 Orlando, FL                         52.58   27034.86
                                          275 Tampa, FL                           85.01   27119.87
                                          234 Saint Petersburg, FL                19.54   27139.41
                                          247 Sarasota, FL                        31.73   27171.14
                                          296 West Palm Beach, FL                176.46    27347.6
                                          164 Miami, FL                           65.75   27413.34
                                          136 Key West, FL                       138.36    27551.7
                                          274 Tallahassee, FL                    441.61   27993.31
                                          195 Pensacola, FL                      202.87   28196.18
                                          168 Mobile, AL                          60.12    28256.3
                                           28 Biloxi, MS                          61.74   28318.04
                                          112 Gulfport, MS                        14.47   28332.51
                                          180 New Orleans, LA                     73.62   28406.13
                                           20 Baton Rouge, LA                     82.09   28488.22
                                          176 Natchez, MS                         78.58    28566.8
                                          126 Jackson, MS                         98.43   28665.22
                                           56 Chattanooga, TN                    386.64   29051.86
                                          303 Wilmington, NC                     512.01   29563.88
                                          220 Richmond, VA                       232.37   29796.25
                                           41 Buffalo, NY                        381.27   30177.52
                                          182 Niagara Falls, ON                   20.54   30198.06
                                          227 Saint Catherines, ON                 8.92   30206.99
                                          223 Rochester, NY                      107.53   30314.52
                                          185 North Bay, ON                      260.14   30574.66
                                          269 Sudbury, ON                            93   30667.66
                                          279 Timmins, ON                        140.17   30807.83
                                          249 Sault Ste Marie, ON                260.02   31067.85
                                          278 Thunder Bay, ON                    357.29   31425.14
                                           75 Des Moines, IA                     553.68   31978.82
                                           84 El Paso, TX                       1119.89   33098.72
                                          144 Las Vegas, NV                      671.08    33769.8
                                          209 Prince Rupert, BC                 1638.53   35408.32
                                          131 Juneau, AK                         390.43   35798.75
                                          298 Whitehorse, YK                     172.32   35971.07
                                           70 Dawson, YT                         362.92   36333.99
                                           90 Fairbanks, AK                         596      36930
                                            8 Anchorage, AK                      292.11    37222.1
                                          183 Nome, AK                          1095.15   38317.26
                                          148 Lihue, HI                          2967.4   41284.65
                                          120 Honolulu, HI                        114.5   41399.15
                                          119 Hilo, HI                              176   41575.15
                                          309 Yellowknife, NT                   4111.93   45687.09
                                           59 Churchill, MB                     1431.71    47118.8
                                            5 Alert, NT                         2742.38   49861.18
                                          243 San Juan, PR                      4433.43   54294.61
                                          184 Norfolk, VA                       1451.64   55746.24
51 -> 288             1   56794.75         51 Champaign, IL
                                          288 Urbana, IL                           2.52       2.52
                                           73 Decatur, IL                         54.92      57.45
                                          262 Springfield, IL                     47.66     105.11
                                          196 Peoria, IL                          61.75     166.86
                                           32 Bloomington, IL                      43.6     210.46
                                          224 Rockford, IL                       123.66     334.12
                                          135 Kenosha, WI                         90.57     424.69
                                          165 Milwaukee, WI                       31.93     456.62
                                          214 Racine, WI                          23.24     479.86
                                           58 Chicago, IL                         61.23     541.09
                                          104 Gary, IN                            27.47     568.56
                                          259 South Bend, IN                      76.01     644.57
                                          132 Kalamazoo, MI                       62.16     706.73
                                           21 Battle Creek, MI                    28.23     734.96
                                          142 Lansing, MI                         51.65     786.61
                                          125 Jackson, MI                         35.28     821.89
                                            9 Ann Arbor, MI                       46.67     868.56
                                          280 Toledo, OH                          43.57     912.13
                                           76 Detroit, MI                         58.03     970.16
                                          304 Windsor, ON                          7.21     977.37
                                           94 Flint, MI                           71.67    1049.04
                                          226 Saginaw, MI                          33.5    1082.54
                                           22 Bay City, MI                        12.83    1095.36
                                          106 Grand Rapids, MI                   130.44     1225.8
                                           96 Ft Wayne, IN                          132     1357.8
                                          174 Muncie, IN                          67.15    1424.95
                                          123 Indianapolis, IN                    60.87    1485.83
                                          140 Lafayette, IN                       66.81    1552.63
                                          276 Terre Haute, IN                     75.45    1628.09
                                           89 Evansville, IN                     103.56    1731.64
                                          192 Paducah, KY                         94.86    1826.51
                                          175 Nashville, TN                      140.56    1967.07
                                           35 Bowling Green, KY                   61.65    2028.71
                                          154 Louisville, KY                       99.3    2128.02
                                          147 Lexington, KY                       90.44    2218.46
                                           60 Cincinnati, OH                      81.08    2299.54
                                          114 Hamilton, OH                        17.93    2317.47
                                           71 Dayton, OH                          35.63     2353.1
                                          265 Springfield, OH                     28.81    2381.91
                                           66 Columbus, OH                        56.03    2437.94
                                          312 Zanesville, OH                      68.11    2506.04
                                           47 Canton, OH                          73.78    2579.83
                                            2 Akron, OH                            21.8    2601.63
                                           61 Cleveland, OH                       31.35    2632.98
                                          310 Youngstown, OH                      77.36    2710.33
                                          267 Steubenville, OH                    50.45    2760.79
                                          297 Wheeling, WV                        21.97    2782.75
                                          201 Pittsburgh, PA                      56.46    2839.21
                                           86 Erie, PA                           116.83    2956.05
                                           37 Brantford, ON                       71.49    3027.54
                                          138 Kitchener, ON                       22.82    3050.35
                                          111 Guelph, ON                          10.29    3060.65
                                           42 Burlington, ONT                     29.68    3090.32
                                          115 Hamilton, ON                        19.51    3109.83
                                          227 Saint Catherines, ON                52.51    3162.34
                                           41 Buffalo, NY                         29.25    3191.58
                                          182 Niagara Falls, ON                   20.54    3212.12
                                          282 Toronto, ON                         42.01    3254.13
                                          197 Peterborough, ON                    82.62    3336.75
                                           24 Belleville, ON                      72.87    3409.61
                                          137 Kingston, ON                        49.28     3458.9
                                          191 Ottawa, ON                          85.39    3544.28
                                          172 Montreal, QC                       146.68    3690.97
                                           43 Burlington, VT                      97.19    3788.16
                                          171 Montpelier, VT                      46.45    3834.61
                                           38 Brattleboro, VT                     97.37    3931.97
                                          263 Springfield, MA                     51.84    3983.81
                                          117 Hartford, CT                        24.25    4008.06
                                          178 New Britain,CT                       9.62    4017.69
                                          163 Meriden, CT                          8.71     4026.4
                                          179 New Haven, CT                       17.96    4044.36
                                           39 Bridgeport, CT                      21.46    4065.82
                                          266 Stamford, CT                        24.55    4090.37
                                          299 White Plains, NY                    15.36    4105.73
                                          181 New York, NY                        27.75    4133.48
                                          128 Jersey City, NJ                      5.04    4138.52
                                          177 Newark, NJ                           6.57    4145.09
                                           83 Elizabeth, NJ                        5.61     4150.7
                                          194 Paterson, NJ                        17.67    4168.37
                                          283 Trenton, NJ                          62.4    4230.77
                                          198 Philadelphia, PA                    34.35    4265.12
                                          302 Wilmington, DE                      30.05    4295.18
                                          217 Reading, PA                         48.48    4343.66
                                          129 Johnstown, PA                        27.2    4370.86
                                          141 Lancaster, PA                       16.64     4387.5
                                          116 Harrisburg, PA                      43.19    4430.69
                                           18 Baltimore, MD                        70.5    4501.18
                                          294 Washington, DC                      40.06    4541.24
                                          220 Richmond, VA                        97.21    4638.45
                                          208 Portsmouth, VA                      94.39    4732.84
                                          184 Norfolk, VA                          1.19    4734.03
                                          215 Raleigh, NC                        178.76    4912.79
                                           80 Durham, NC                          23.61    4936.41
                                          109 Greensboro, NC                      61.97    4998.37
                                          306 Winston-Salem, NC                    31.3    5029.67
                                           54 Charlotte, NC                       73.13     5102.8
                                          260 Spartanburg, NC                     77.64    5180.45
                                          110 Greenville, SC                      32.62    5213.06
                                           10 Asheville, NC                       52.87    5265.93
                                          139 Knoxville, TN                       97.65    5363.58
                                           56 Chattanooga, TN                    114.91    5478.49
                                          100 Gadsden, AL                         86.02    5564.51
                                          122 Huntsville, AL                      63.64    5628.15
                                           30 Birmingham, AL                      84.91    5713.06
                                          170 Montgomery, AL                      86.96    5800.02
                                           65 Columbus, GA                         90.9    5890.93
                                          157 Macon, GA                           97.26    5988.18
                                           12 Atlanta, GA                         81.64    6069.82
                                           14 Augusta, GA                        167.83    6237.65
                                           64 Columbia, SC                        74.55    6312.21
                                           52 Charleston, SC                      113.9     6426.1
                                          250 Savannah, GA                         93.9       6520
                                          127 Jacksonville, FL                   126.96    6646.96
                                          101 Gainesville, FL                     65.95    6712.91
                                           72 Daytona Beach, FL                   94.98    6807.89
                                          190 Orlando, FL                         52.58    6860.48
                                          275 Tampa, FL                           85.01    6945.48
                                          234 Saint Petersburg, FL                19.54    6965.02
                                          247 Sarasota, FL                        31.73    6996.75
                                          296 West Palm Beach, FL                176.46    7173.21
                                          164 Miami, FL                           65.75    7238.96
                                          136 Key West, FL                       138.36    7377.32
                                          274 Tallahassee, FL                    441.61    7818.92
                                          195 Pensacola, FL                      202.87     8021.8
                                          168 Mobile, AL                          60.12    8081.91
                                           28 Biloxi, MS                          61.74    8143.65
                                          112 Gulfport, MS                        14.47    8158.12
                                          180 New Orleans, LA                     73.62    8231.74
                                           20 Baton Rouge, LA                     82.09    8313.83
                                          176 Natchez, MS                         78.58    8392.41
                                          126 Jackson, MS                         98.43    8490.84
                                          162 Memphis, TN                         197.2    8688.04
                                          151 Little Rock, AR                     157.3    8845.33
                                          277 Texarkana, TX                      151.96    8997.29
                                          160 Marshall, TX                        64.71       9062
                                          256 Shreveport, LA                      42.67    9104.67
                                           23 Beaumont, TX                       170.28    9274.96
                                          204 Port Arthur, TX                     17.62    9292.58
                                          103 Galveston, TX                       72.86    9365.44
                                          121 Houston, TX                         50.45    9415.89
                                           16 Austin, TX                         168.07    9583.96
                                          238 San Antonio, TX                     77.99    9661.95
                                           68 Corpus Christi, TX                 135.39    9797.35
                                          143 Laredo, TX                         147.28    9944.63
                                          292 Waco, TX                            323.5   10268.13
                                           97 Ft Worth, TX                        82.15   10350.28
                                           69 Dallas, TX                          36.19   10386.47
                                          188 Oklahoma City, OK                  191.95   10578.42
                                           85 Enid, OK                            68.83   10647.25
                                          300 Wichita, KS                         97.07   10744.32
                                          236 Salina, KS                          81.55   10825.87
                                          281 Topeka, KS                         134.35   10960.22
                                          133 Kansas City, KS                     72.73   11032.95
                                          134 Kansas City, MO                      3.52   11036.48
                                          231 Saint Joseph, MO                    49.79   11086.27
                                          189 Omaha, NE                          127.61   11213.88
                                          150 Lincoln, NE                         59.52    11273.4
                                          257 Sioux City, IA                     118.91    11392.3
                                          258 Sioux Falls, SD                     75.45   11467.76
                                          228 Saint Cloud, MN                    223.72   11691.48
                                          166 Minneapolis, MN                     73.94   11765.42
                                          233 Saint Paul, MN                      12.03   11777.45
                                          222 Rochester, MN                       76.94   11854.39
                                           81 Eau Claire, WI                       86.5   11940.89
                                          270 Superior, WI                       138.42    12079.3
                                           79 Duluth, MN                           4.32   12083.63
                                          278 Thunder Bay, ON                    221.38   12305.01
                                          108 Green Bay, WI                      281.05   12586.06
                                          253 Sheboygan, WI                       57.13   12643.19
                                          158 Madison, WI                         125.6   12768.78
                                           78 Dubuque, IA                         95.84   12864.62
                                           49 Cedar Rapids, IA                    75.74   12940.36
                                          124 Iowa City, IA                       25.25   12965.61
                                          295 Waterloo, IA                        80.35   13045.96
                                           75 Des Moines, IA                     107.02   13152.99
                                           63 Columbia, MO                       203.13   13356.12
                                          264 Springfield, MO                    137.23   13493.35
                                          130 Joplin, MO                          84.44   13577.79
                                           95 Ft Smith, AR                       117.62    13695.4
                                          287 Tulsa, OK                          122.27   13817.67
                                           77 Dodge City, KS                     299.19   14116.86
                                            7 Amarillo, TX                       215.16   14332.02
                                          156 Lubbock, TX                        113.61   14445.63
                                            1 Abilene, TX                        166.25   14611.88
                                           84 El Paso, TX                        469.27   15081.15
                                            4 Albuquerque, NM                    230.08   15311.23
                                          246 Santa Fe, NM                        64.52   15375.75
                                          102 Gallup, NM                         194.11   15569.86
                                           93 Flagstaff, AZ                      202.26   15772.12
                                          199 Phoenix, AZ                        124.38    15896.5
                                          286 Tucson, AZ                         116.06   16012.56
                                          311 Yuma, AZ                           257.86   16270.41
                                          240 San Diego, CA                      175.01   16445.43
                                          239 San Bernardino, CA                  96.69   16542.12
                                          193 Pasadena, CA                        59.12   16601.24
                                          153 Los Angeles, CA                      9.51   16610.75
                                          244 Santa Barbara, CA                  103.68   16714.43
                                           17 Bakersfield, CA                     80.84   16795.27
                                           99 Fresno, CA                         108.31   16903.58
                                          268 Stockton, CA                       134.15   17037.73
                                          225 Sacramento, CA                      45.35   17083.08
                                           26 Berkeley, CA                         72.8   17155.88
                                          186 Oakland, CA                          4.65   17160.53
                                          241 San Francisco, CA                   10.47      17171
                                          242 San Jose, CA                        47.11   17218.11
                                          245 Santa Cruz, CA                      26.93   17245.03
                                           48 Carson City, NV                    217.59   17462.63
                                          219 Reno, NV                            25.48   17488.11
                                           88 Eureka, CA                         313.15   17801.26
                                           87 Eugene, OR                         236.57   18037.82
                                          235 Salem, OR                           61.63   18099.46
                                          206 Portland, OR                        47.17   18146.63
                                          273 Tacoma, WA                         120.57    18267.2
                                          252 Seattle, WA                         25.62   18292.82
                                           25 Bellingham, WA                      80.42   18373.24
                                          290 Vancouver, BC                       56.66    18429.9
                                          291 Victoria, BC                        41.45   18471.35
                                          308 Yakima, WA                         246.37   18717.72
                                          293 Walla Walla, WA                    153.98    18871.7
                                          261 Spokane, WA                        127.07   18998.77
                                           33 Boise, ID                          291.99   19290.76
                                          203 Pocatello, ID                      264.67   19555.43
                                          187 Ogden, UT                          118.47    19673.9
                                          237 Salt Lake City, UT                  32.44   19706.34
                                          211 Provo, UT                           39.79   19746.13
                                          105 Grand Junction, CO                 229.45   19975.58
                                           74 Denver, CO                         250.76   20226.34
                                           62 Colorado Springs, CO                63.56    20289.9
                                          212 Pueblo, CO                          42.63   20332.53
                                           57 Cheyenne, WY                       199.91   20532.45
                                          216 Rapid City, SD                     230.96    20763.4
                                          200 Pierre, SD                         199.98   20963.38
                                           31 Bismarck, ND                       171.22    21134.6
                                          167 Minot, ND                          104.59   21239.19
                                           36 Brandon, MB                        148.55   21387.74
                                          305 Winnipeg, MB                       187.08   21574.82
                                           92 Fargo, ND                          211.94   21786.76
                                          218 Regina, SK                         597.09   22383.85
                                          173 Moose Jaw, SK                       62.04   22445.89
                                          248 Saskatoon, SK                      149.68   22595.57
                                          161 Medicine Hat, AB                   315.13    22910.7
                                          146 Lethbridge, AB                     146.13   23056.82
                                           45 Calgary, AB                        132.75   23189.57
                                           82 Edmonton, AB                       173.35   23362.92
                                          107 Great Falls, MT                    446.74   23809.67
                                          118 Helena, MT                           80.7   23890.36
                                           44 Butte, MT                           53.31   23943.68
                                           27 Billings, MT                       279.14   24222.81
                                          255 Sheridan, WY                       126.61   24349.42
                                          144 Las Vegas, NV                      821.24   25170.67
                                          209 Prince Rupert, BC                 1638.53   26809.19
                                          131 Juneau, AK                         390.43   27199.62
                                          298 Whitehorse, YK                     172.32   27371.94
                                           70 Dawson, YT                         362.92   27734.86
                                           90 Fairbanks, AK                         596   28330.87
                                            8 Anchorage, AK                      292.11   28622.97
                                          183 Nome, AK                          1095.15   29718.13
                                          148 Lihue, HI                          2967.4   32685.52
                                          120 Honolulu, HI                        114.5   32800.02
                                          119 Hilo, HI                              176   32976.02
                                          309 Yellowknife, NT                   4111.93   37087.96
                                           59 Churchill, MB                     1431.71   38519.67
                                          249 Sault Ste Marie, ON               1073.27   39592.94
                                          269 Sudbury, ON                        256.57   39849.51
                                          185 North Bay, ON                          93   39942.51
                                          279 Timmins, ON                        198.01   40140.52
                                          152 London, ON                         387.85   40528.37
                                          223 Rochester, NY                       249.8   40778.17
                                          272 Syracuse, NY                       101.71   40879.88
                                          289 Utica, NY                           63.31   40943.18
                                           29 Binghamtom, NY                      83.89   41027.07
                                          301 Wilkes-Barre, PA                    58.97   41086.05
                                            6 Allentown, PA                       51.68   41137.72
                                           13 Atlantic City, NJ                  113.26   41250.99
                                           50 Central Islip, NY                  129.74   41380.73
                                          202 Pittsfield, MA                      114.7   41495.43
                                          285 Troy, NY                            36.34   41531.77
                                            3 Albany, NY                           6.88   41538.65
                                          251 Schenectady, NY                     16.88   41555.54
                                          307 Worcester, MA                      152.51   41708.04
                                          210 Providence, RI                      40.53   41748.58
                                           91 Fall River, MA                      19.72   41768.29
                                           40 Brockton, MA                        28.03   41796.32
                                           34 Boston, MA                          19.21   41815.53
                                           46 Cambridge, MA                        3.36    41818.9
                                          155 Lowell, MA                          23.03   41841.93
                                          145 Lawrence, MA                        11.73   41853.65
                                          159 Manchester, NH                      28.36   41882.01
                                           67 Concord, NH                         15.76   41897.76
                                          207 Portsmouth, NH                      54.37   41952.14
                                          205 Portland, ME                        53.74   42005.88
                                           15 Augusta, ME                         55.61   42061.49
                                           19 Bangor, ME                          77.07   42138.56
                                           98 Fredericton, NB                    171.89   42310.45
                                          229 Saint John, NB                      58.53   42368.98
                                          169 Moncton, NB                         99.94   42468.93
                                           55 Charlottetown, PE                  108.58    42577.5
                                          113 Halifax, NS                        100.98   42678.49
                                          271 Syndey, NS                         254.97   42933.46
                                          230 Saint John's, NF                   514.07   43447.53
                                          213 Quebec City, QC                   1290.47   44737.99
                                          284 Trois-Rivieres, QC                 190.67   44928.66
                                          254 Sherbrooke, QC                      93.08   45021.74
                                          221 Roanoke, VA                        802.17   45823.91
                                           53 Charleston, WV                     138.61   45962.52
                                           11 Ashland, KY                         70.03   46032.55
                                          149 Lima, OH                           186.42   46218.96
                                          232 Saint Louis, MO                    445.62   46664.59
                                          303 Wilmington, NC                     899.59   47564.17
                                          243 San Juan, PR                      1361.81   48925.99
                                            5 Alert, NT                         4433.43   53359.42
                                           51 Champaign, IL                     3435.34   56794.75
                      2   56794.89         51 Champaign, IL
                                          288 Urbana, IL                           2.52       2.52
                                           73 Decatur, IL                         54.92      57.45
                                          262 Springfield, IL                     47.66     105.11
                                          196 Peoria, IL                          61.75     166.86
                                           32 Bloomington, IL                      43.6     210.46
                                          224 Rockford, IL                       123.66     334.12
                                          135 Kenosha, WI                         90.57     424.69
                                          165 Milwaukee, WI                       31.93     456.62
                                          214 Racine, WI                          23.24     479.86
                                           58 Chicago, IL                         61.23     541.09
                                          104 Gary, IN                            27.47     568.56
                                          259 South Bend, IN                      76.01     644.57
                                          132 Kalamazoo, MI                       62.16     706.73
                                           21 Battle Creek, MI                    28.23     734.96
                                          142 Lansing, MI                         51.65     786.61
                                          125 Jackson, MI                         35.28     821.89
                                            9 Ann Arbor, MI                       46.67     868.56
                                          280 Toledo, OH                          43.57     912.13
                                           76 Detroit, MI                         58.03     970.16
                                          304 Windsor, ON                          7.21     977.37
                                           94 Flint, MI                           71.67    1049.04
                                          226 Saginaw, MI                          33.5    1082.54
                                           22 Bay City, MI                        12.83    1095.36
                                          106 Grand Rapids, MI                   130.44     1225.8
                                           96 Ft Wayne, IN                          132     1357.8
                                          174 Muncie, IN                          67.15    1424.95
                                          123 Indianapolis, IN                    60.87    1485.83
                                          140 Lafayette, IN                       66.81    1552.63
                                          276 Terre Haute, IN                     75.45    1628.09
                                           89 Evansville, IN                     103.56    1731.64
                                          192 Paducah, KY                         94.86    1826.51
                                          175 Nashville, TN                      140.56    1967.07
                                           35 Bowling Green, KY                   61.65    2028.71
                                          154 Louisville, KY                       99.3    2128.02
                                          147 Lexington, KY                       90.44    2218.46
                                           60 Cincinnati, OH                      81.08    2299.54
                                          114 Hamilton, OH                        17.93    2317.47
                                           71 Dayton, OH                          35.63     2353.1
                                          265 Springfield, OH                     28.81    2381.91
                                           66 Columbus, OH                        56.03    2437.94
                                          312 Zanesville, OH                      68.11    2506.04
                                           47 Canton, OH                          73.78    2579.83
                                            2 Akron, OH                            21.8    2601.63
                                           61 Cleveland, OH                       31.35    2632.98
                                          310 Youngstown, OH                      77.36    2710.33
                                          267 Steubenville, OH                    50.45    2760.79
                                          297 Wheeling, WV                        21.97    2782.75
                                          201 Pittsburgh, PA                      56.46    2839.21
                                           86 Erie, PA                           116.83    2956.05
                                           37 Brantford, ON                       71.49    3027.54
                                          138 Kitchener, ON                       22.82    3050.35
                                          111 Guelph, ON                          10.29    3060.65
                                           42 Burlington, ONT                     29.68    3090.32
                                          115 Hamilton, ON                        19.51    3109.83
                                          227 Saint Catherines, ON                52.51    3162.34
                                          182 Niagara Falls, ON                    8.92    3171.26
                                           41 Buffalo, NY                         20.54     3191.8
                                          282 Toronto, ON                         62.46    3254.26
                                          197 Peterborough, ON                    82.62    3336.88
                                           24 Belleville, ON                      72.87    3409.75
                                          137 Kingston, ON                        49.28    3459.03
                                          191 Ottawa, ON                          85.39    3544.41
                                          172 Montreal, QC                       146.68     3691.1
                                           43 Burlington, VT                      97.19    3788.29
                                          171 Montpelier, VT                      46.45    3834.74
                                           38 Brattleboro, VT                     97.37    3932.11
                                          263 Springfield, MA                     51.84    3983.94
                                          117 Hartford, CT                        24.25     4008.2
                                          178 New Britain,CT                       9.62    4017.82
                                          163 Meriden, CT                          8.71    4026.53
                                          179 New Haven, CT                       17.96    4044.49
                                           39 Bridgeport, CT                      21.46    4065.95
                                          266 Stamford, CT                        24.55     4090.5
                                          299 White Plains, NY                    15.36    4105.86
                                          181 New York, NY                        27.75    4133.61
                                          128 Jersey City, NJ                      5.04    4138.65
                                          177 Newark, NJ                           6.57    4145.22
                                           83 Elizabeth, NJ                        5.61    4150.84
                                          194 Paterson, NJ                        17.67     4168.5
                                          283 Trenton, NJ                          62.4     4230.9
                                          198 Philadelphia, PA                    34.35    4265.26
                                          302 Wilmington, DE                      30.05    4295.31
                                          217 Reading, PA                         48.48    4343.79
                                          129 Johnstown, PA                        27.2    4370.99
                                          141 Lancaster, PA                       16.64    4387.63
                                          116 Harrisburg, PA                      43.19    4430.82
                                           18 Baltimore, MD                        70.5    4501.31
                                          294 Washington, DC                      40.06    4541.38
                                          220 Richmond, VA                        97.21    4638.58
                                          208 Portsmouth, VA                      94.39    4732.97
                                          184 Norfolk, VA                          1.19    4734.16
                                          215 Raleigh, NC                        178.76    4912.92
                                           80 Durham, NC                          23.61    4936.54
                                          109 Greensboro, NC                      61.97    4998.51
                                          306 Winston-Salem, NC                    31.3     5029.8
                                           54 Charlotte, NC                       73.13    5102.94
                                          260 Spartanburg, NC                     77.64    5180.58
                                          110 Greenville, SC                      32.62    5213.19
                                           10 Asheville, NC                       52.87    5266.06
                                          139 Knoxville, TN                       97.65    5363.71
                                           56 Chattanooga, TN                    114.91    5478.63
                                          100 Gadsden, AL                         86.02    5564.64
                                          122 Huntsville, AL                      63.64    5628.29
                                           30 Birmingham, AL                      84.91    5713.19
                                          170 Montgomery, AL                      86.96    5800.16
                                           65 Columbus, GA                         90.9    5891.06
                                          157 Macon, GA                           97.26    5988.32
                                           12 Atlanta, GA                         81.64    6069.95
                                           14 Augusta, GA                        167.83    6237.79
                                           64 Columbia, SC                        74.55    6312.34
                                           52 Charleston, SC                      113.9    6426.24
                                          250 Savannah, GA                         93.9    6520.14
                                          127 Jacksonville, FL                   126.96    6647.09
                                          101 Gainesville, FL                     65.95    6713.05
                                           72 Daytona Beach, FL                   94.98    6808.02
                                          190 Orlando, FL                         52.58    6860.61
                                          275 Tampa, FL                           85.01    6945.62
                                          234 Saint Petersburg, FL                19.54    6965.15
                                          247 Sarasota, FL                        31.73    6996.88
                                          296 West Palm Beach, FL                176.46    7173.34
                                          164 Miami, FL                           65.75    7239.09
                                          136 Key West, FL                       138.36    7377.45
                                          274 Tallahassee, FL                    441.61    7819.06
                                          195 Pensacola, FL                      202.87    8021.93
                                          168 Mobile, AL                          60.12    8082.04
                                           28 Biloxi, MS                          61.74    8143.79
                                          112 Gulfport, MS                        14.47    8158.25
                                          180 New Orleans, LA                     73.62    8231.87
                                           20 Baton Rouge, LA                     82.09    8313.96
                                          176 Natchez, MS                         78.58    8392.54
                                          126 Jackson, MS                         98.43    8490.97
                                          162 Memphis, TN                         197.2    8688.17
                                          151 Little Rock, AR                     157.3    8845.47
                                          277 Texarkana, TX                      151.96    8997.42
                                          160 Marshall, TX                        64.71    9062.13
                                          256 Shreveport, LA                      42.67    9104.81
                                           23 Beaumont, TX                       170.28    9275.09
                                          204 Port Arthur, TX                     17.62    9292.71
                                          103 Galveston, TX                       72.86    9365.57
                                          121 Houston, TX                         50.45    9416.02
                                           16 Austin, TX                         168.07     9584.1
                                          238 San Antonio, TX                     77.99    9662.09
                                           68 Corpus Christi, TX                 135.39    9797.48
                                          143 Laredo, TX                         147.28    9944.76
                                          292 Waco, TX                            323.5   10268.26
                                           97 Ft Worth, TX                        82.15   10350.41
                                           69 Dallas, TX                          36.19    10386.6
                                          188 Oklahoma City, OK                  191.95   10578.55
                                           85 Enid, OK                            68.83   10647.38
                                          300 Wichita, KS                         97.07   10744.46
                                          236 Salina, KS                          81.55      10826
                                          281 Topeka, KS                         134.35   10960.35
                                          133 Kansas City, KS                     72.73   11033.09
                                          134 Kansas City, MO                      3.52   11036.61
                                          231 Saint Joseph, MO                    49.79    11086.4
                                          189 Omaha, NE                          127.61   11214.01
                                          150 Lincoln, NE                         59.52   11273.53
                                          257 Sioux City, IA                     118.91   11392.44
                                          258 Sioux Falls, SD                     75.45   11467.89
                                          228 Saint Cloud, MN                    223.72   11691.61
                                          166 Minneapolis, MN                     73.94   11765.55
                                          233 Saint Paul, MN                      12.03   11777.58
                                          222 Rochester, MN                       76.94   11854.52
                                           81 Eau Claire, WI                       86.5   11941.02
                                          270 Superior, WI                       138.42   12079.44
                                           79 Duluth, MN                           4.32   12083.76
                                          278 Thunder Bay, ON                    221.38   12305.14
                                          108 Green Bay, WI                      281.05   12586.19
                                          253 Sheboygan, WI                       57.13   12643.32
                                          158 Madison, WI                         125.6   12768.91
                                           78 Dubuque, IA                         95.84   12864.75
                                           49 Cedar Rapids, IA                    75.74    12940.5
                                          124 Iowa City, IA                       25.25   12965.74
                                          295 Waterloo, IA                        80.35    13046.1
                                           75 Des Moines, IA                     107.02   13153.12
                                           63 Columbia, MO                       203.13   13356.25
                                          264 Springfield, MO                    137.23   13493.48
                                          130 Joplin, MO                          84.44   13577.92
                                           95 Ft Smith, AR                       117.62   13695.54
                                          287 Tulsa, OK                          122.27    13817.8
                                           77 Dodge City, KS                     299.19      14117
                                            7 Amarillo, TX                       215.16   14332.15
                                          156 Lubbock, TX                        113.61   14445.76
                                            1 Abilene, TX                        166.25   14612.02
                                           84 El Paso, TX                        469.27   15081.29
                                            4 Albuquerque, NM                    230.08   15311.37
                                          246 Santa Fe, NM                        64.52   15375.88
                                          102 Gallup, NM                         194.11   15569.99
                                           93 Flagstaff, AZ                      202.26   15772.25
                                          199 Phoenix, AZ                        124.38   15896.63
                                          286 Tucson, AZ                         116.06   16012.69
                                          311 Yuma, AZ                           257.86   16270.55
                                          240 San Diego, CA                      175.01   16445.56
                                          239 San Bernardino, CA                  96.69   16542.25
                                          193 Pasadena, CA                        59.12   16601.38
                                          153 Los Angeles, CA                      9.51   16610.89
                                          244 Santa Barbara, CA                  103.68   16714.56
                                           17 Bakersfield, CA                     80.84   16795.41
                                           99 Fresno, CA                         108.31   16903.71
                                          268 Stockton, CA                       134.15   17037.87
                                          225 Sacramento, CA                      45.35   17083.21
                                           26 Berkeley, CA                         72.8   17156.01
                                          186 Oakland, CA                          4.65   17160.66
                                          241 San Francisco, CA                   10.47   17171.13
                                          242 San Jose, CA                        47.11   17218.24
                                          245 Santa Cruz, CA                      26.93   17245.17
                                           48 Carson City, NV                    217.59   17462.76
                                          219 Reno, NV                            25.48   17488.24
                                           88 Eureka, CA                         313.15   17801.39
                                           87 Eugene, OR                         236.57   18037.96
                                          235 Salem, OR                           61.63   18099.59
                                          206 Portland, OR                        47.17   18146.76
                                          273 Tacoma, WA                         120.57   18267.33
                                          252 Seattle, WA                         25.62   18292.95
                                           25 Bellingham, WA                      80.42   18373.37
                                          290 Vancouver, BC                       56.66   18430.03
                                          291 Victoria, BC                        41.45   18471.48
                                          308 Yakima, WA                         246.37   18717.85
                                          293 Walla Walla, WA                    153.98   18871.83
                                          261 Spokane, WA                        127.07    18998.9
                                           33 Boise, ID                          291.99    19290.9
                                          203 Pocatello, ID                      264.67   19555.56
                                          187 Ogden, UT                          118.47   19674.03
                                          237 Salt Lake City, UT                  32.44   19706.47
                                          211 Provo, UT                           39.79   19746.26
                                          105 Grand Junction, CO                 229.45   19975.72
                                           74 Denver, CO                         250.76   20226.47
                                           62 Colorado Springs, CO                63.56   20290.03
                                          212 Pueblo, CO                          42.63   20332.67
                                           57 Cheyenne, WY                       199.91   20532.58
                                          216 Rapid City, SD                     230.96   20763.54
                                          200 Pierre, SD                         199.98   20963.52
                                           31 Bismarck, ND                       171.22   21134.74
                                          167 Minot, ND                          104.59   21239.32
                                           36 Brandon, MB                        148.55   21387.87
                                          305 Winnipeg, MB                       187.08   21574.95
                                           92 Fargo, ND                          211.94   21786.89
                                          218 Regina, SK                         597.09   22383.98
                                          173 Moose Jaw, SK                       62.04   22446.03
                                          248 Saskatoon, SK                      149.68    22595.7
                                          161 Medicine Hat, AB                   315.13   22910.83
                                          146 Lethbridge, AB                     146.13   23056.96
                                           45 Calgary, AB                        132.75    23189.7
                                           82 Edmonton, AB                       173.35   23363.06
                                          107 Great Falls, MT                    446.74    23809.8
                                          118 Helena, MT                           80.7    23890.5
                                           44 Butte, MT                           53.31   23943.81
                                           27 Billings, MT                       279.14   24222.95
                                          255 Sheridan, WY                       126.61   24349.56
                                          144 Las Vegas, NV                      821.24    25170.8
                                          209 Prince Rupert, BC                 1638.53   26809.33
                                          131 Juneau, AK                         390.43   27199.75
                                          298 Whitehorse, YK                     172.32   27372.07
                                           70 Dawson, YT                         362.92      27735
                                           90 Fairbanks, AK                         596      28331
                                            8 Anchorage, AK                      292.11   28623.11
                                          183 Nome, AK                          1095.15   29718.26
                                          148 Lihue, HI                          2967.4   32685.65
                                          120 Honolulu, HI                        114.5   32800.16
                                          119 Hilo, HI                              176   32976.16
                                          309 Yellowknife, NT                   4111.93   37088.09
                                           59 Churchill, MB                     1431.71    38519.8
                                          249 Sault Ste Marie, ON               1073.27   39593.07
                                          269 Sudbury, ON                        256.57   39849.65
                                          185 North Bay, ON                          93   39942.65
                                          279 Timmins, ON                        198.01   40140.66
                                          152 London, ON                         387.85   40528.51
                                          223 Rochester, NY                       249.8    40778.3
                                          272 Syracuse, NY                       101.71   40880.01
                                          289 Utica, NY                           63.31   40943.32
                                           29 Binghamtom, NY                      83.89   41027.21
                                          301 Wilkes-Barre, PA                    58.97   41086.18
                                            6 Allentown, PA                       51.68   41137.86
                                           13 Atlantic City, NJ                  113.26   41251.12
                                           50 Central Islip, NY                  129.74   41380.86
                                          202 Pittsfield, MA                      114.7   41495.56
                                          285 Troy, NY                            36.34    41531.9
                                            3 Albany, NY                           6.88   41538.78
                                          251 Schenectady, NY                     16.88   41555.67
                                          307 Worcester, MA                      152.51   41708.18
                                          210 Providence, RI                      40.53   41748.71
                                           91 Fall River, MA                      19.72   41768.43
                                           40 Brockton, MA                        28.03   41796.45
                                           34 Boston, MA                          19.21   41815.67
                                           46 Cambridge, MA                        3.36   41819.03
                                          155 Lowell, MA                          23.03   41842.06
                                          145 Lawrence, MA                        11.73   41853.79
                                          159 Manchester, NH                      28.36   41882.14
                                           67 Concord, NH                         15.76    41897.9
                                          207 Portsmouth, NH                      54.37   41952.27
                                          205 Portland, ME                        53.74   42006.01
                                           15 Augusta, ME                         55.61   42061.62
                                           19 Bangor, ME                          77.07    42138.7
                                           98 Fredericton, NB                    171.89   42310.58
                                          229 Saint John, NB                      58.53   42369.11
                                          169 Moncton, NB                         99.94   42469.06
                                           55 Charlottetown, PE                  108.58   42577.64
                                          113 Halifax, NS                        100.98   42678.62
                                          271 Syndey, NS                         254.97   42933.59
                                          230 Saint John's, NF                   514.07   43447.66
                                          213 Quebec City, QC                   1290.47   44738.13
                                          284 Trois-Rivieres, QC                 190.67    44928.8
                                          254 Sherbrooke, QC                      93.08   45021.87
                                          221 Roanoke, VA                        802.17   45824.05
                                           53 Charleston, WV                     138.61   45962.65
                                           11 Ashland, KY                         70.03   46032.68
                                          149 Lima, OH                           186.42    46219.1
                                          232 Saint Louis, MO                    445.62   46664.72
                                          303 Wilmington, NC                     899.59   47564.31
                                          243 San Juan, PR                      1361.81   48926.12
                                            5 Alert, NT                         4433.43   53359.55
                                           51 Champaign, IL                     3435.34   56794.89

1252 rows selected.

Elapsed: 00:01:25.25
Longest Routes
==============
ROOT_LEG       PATH_RNK   TOT_DIST    TOWN_ID NAME                             LEG_DIST   CUM_DIST
------------ ---------- ---------- ---------- ------------------------------ ---------- ----------
183 -> 230            1  649684.75        183 Nome, AK
                                          230 Saint John's, NF                  7870.89    7870.89
                                          148 Lihue, HI                         7576.95   15447.85
                                            5 Alert, NT                         7905.76   23353.61
                                          119 Hilo, HI                          7786.39   31139.99
                                          271 Syndey, NS                        6867.36   38007.35
                                          120 Honolulu, HI                      6973.04   44980.39
                                           55 Charlottetown, PE                 6769.69   51750.08
                                            8 Anchorage, AK                     6084.17   57834.25
                                          243 San Juan, PR                       6499.8   64334.05
                                           90 Fairbanks, AK                     6485.59   70819.64
                                          113 Halifax, NS                        5979.6   76799.24
                                           70 Dawson, YT                        5390.11   82189.35
                                          169 Moncton, NB                       5290.95    87480.3
                                          298 Whitehorse, YK                    4966.85   92447.16
                                          229 Saint John, NB                    4896.82   97343.98
                                          131 Juneau, AK                        4819.35  102163.34
                                           98 Fredericton, NB                   4768.78  106932.11
                                          209 Prince Rupert, BC                 4454.24  111386.35
                                           19 Bangor, ME                        4310.99  115697.34
                                           88 Eureka, CA                        3836.77  119534.11
                                           15 Augusta, ME                        3765.4  123299.51
                                          291 Victoria, BC                      3718.78  127018.29
                                          205 Portland, ME                      3689.93  130708.23
                                          290 Vancouver, BC                      3676.9  134385.13
                                          207 Portsmouth, NH                    3646.55  138031.68
                                           87 Eugene, OR                        3615.89  141647.57
                                           40 Brockton, MA                      3600.15  145247.72
                                          235 Salem, OR                         3599.44  148847.16
                                           34 Boston, MA                        3595.59  152442.75
                                           25 Bellingham, WA                    3580.78  156023.53
                                           91 Fall River, MA                    3580.15  159603.68
                                          206 Portland, OR                      3569.55  163173.24
                                           46 Cambridge, MA                     3569.83  166743.06
                                          273 Tacoma, WA                        3563.14   170306.2
                                          145 Lawrence, MA                      3557.08  173863.27
                                          241 San Francisco, CA                 3557.84  177421.11
                                          213 Quebec City, QC                      3607  181028.11
                                          186 Oakland, CA                       3596.53  184624.65
                                          155 Lowell, MA                        3536.42  188161.06
                                          252 Seattle, WA                       3541.57  191702.63
                                          210 Providence, RI                     3540.8  195243.43
                                           26 Berkeley, CA                      3524.68  198768.11
                                          254 Sherbrooke, QC                    3533.82  202301.93
                                          245 Santa Cruz, CA                    3527.03  205828.96
                                          159 Manchester, NH                    3519.15  209348.11
                                          242 San Jose, CA                      3506.93  212855.04
                                           67 Concord, NH                       3502.92  216357.96
                                          225 Sacramento, CA                    3466.46  219824.42
                                          307 Worcester, MA                     3442.81  223267.23
                                          268 Stockton, CA                      3432.24  226699.47
                                          284 Trois-Rivieres, QC                3408.48  230107.95
                                          244 Santa Barbara, CA                 3351.71  233459.66
                                          171 Montpelier, VT                     3326.1  236785.76
                                          308 Yakima, WA                        3315.62  240101.39
                                          263 Springfield, MA                   3325.25  243426.64
                                           99 Fresno, CA                        3280.93  246707.57
                                           38 Brattleboro, VT                   3289.35  249996.92
                                          219 Reno, NV                          3273.12  253270.05
                                          117 Hartford, CT                      3259.94  256529.98
                                           48 Carson City, NV                   3258.04  259788.02
                                          178 New Britain,CT                    3251.14  263039.16
                                           17 Bakersfield, CA                   3224.24   266263.4
                                           43 Burlington, VT                    3226.83  269490.23
                                          153 Los Angeles, CA                   3193.66  272683.89
                                          163 Meriden, CT                        3181.7  275865.59
                                          309 Yellowknife, NT                   3219.55  279085.14
                                          164 Miami, FL                         3467.82  282552.96
                                           82 Edmonton, AB                      2998.65  285551.61
                                          136 Key West, FL                      2971.34  288522.96
                                          261 Spokane, WA                       2934.85  291457.81
                                          179 New Haven, CT                     3105.67  294563.47
                                          193 Pasadena, CA                      3163.09  297726.57
                                          172 Montreal, QC                      3157.19  300883.76
                                          240 San Diego, CA                     3118.23  304001.99
                                          202 Pittsfield, MA                     3107.7  307109.68
                                          293 Walla Walla, WA                   3125.94  310235.63
                                           50 Central Islip, NY                 3140.18   313375.8
                                          239 San Bernardino, CA                3080.98  316456.79
                                           39 Bridgeport, CT                    3084.78  319541.56
                                           33 Boise, ID                         2975.73  322517.29
                                          266 Stamford, CT                      2952.96  325470.25
                                          311 Yuma, AZ                           2896.3  328366.55
                                          285 Troy, NY                          2911.41  331277.97
                                          144 Las Vegas, NV                      2899.2  334177.16
                                            3 Albany, NY                        2893.98  337071.14
                                           45 Calgary, AB                       2842.81  339913.95
                                          296 West Palm Beach, FL               2889.69  342803.64
                                          146 Lethbridge, AB                    2761.59  345565.23
                                          299 White Plains, NY                  2761.87   348327.1
                                           44 Butte, MT                         2700.79  351027.89
                                          181 New York, NY                      2687.05  353714.95
                                          199 Phoenix, AZ                       2677.74  356392.69
                                          251 Schenectady, NY                   2713.16  359105.85
                                          203 Pocatello, ID                     2660.51  361766.36
                                          128 Jersey City, NJ                    2655.1  364421.46
                                          118 Helena, MT                         2653.8  367075.27
                                          177 Newark, NJ                        2647.25  369722.51
                                           93 Flagstaff, AZ                     2617.66  372340.18
                                          194 Paterson, NJ                      2619.56  374959.74
                                          187 Ogden, UT                         2611.96   377571.7
                                           83 Elizabeth, NJ                     2609.47  380181.16
                                          107 Great Falls, MT                   2605.85  382787.01
                                           13 Atlantic City, NJ                  2609.3  385396.31
                                          161 Medicine Hat, AB                  2613.82  388010.13
                                          283 Trenton, NJ                        2576.5  390586.63
                                          237 Salt Lake City, UT                2566.97  393153.59
                                          289 Utica, NY                         2538.01   395691.6
                                          286 Tucson, AZ                        2578.23  398269.82
                                          191 Ottawa, ON                        2575.89  400845.71
                                          211 Provo, UT                          2487.2  403332.91
                                          198 Philadelphia, PA                  2521.63  405854.54
                                           27 Billings, MT                      2338.33  408192.87
                                          302 Wilmington, DE                    2314.81  410507.68
                                          248 Saskatoon, SK                     2317.41  412825.09
                                          247 Sarasota, FL                      2394.44  415219.54
                                           59 Churchill, MB                     2311.24  417530.77
                                          234 Saint Petersburg, FL              2279.52  419810.29
                                          173 Moose Jaw, SK                     2220.06  422030.35
                                          190 Orlando, FL                       2249.04  424279.39
                                          218 Regina, SK                        2208.47  426487.86
                                           72 Daytona Beach, FL                 2195.17  428683.03
                                          255 Sheridan, WY                      2090.58  430773.61
                                            6 Allentown, PA                     2193.28  432966.89
                                          102 Gallup, NM                        2324.19  435291.08
                                           29 Binghamtom, NY                    2312.96  437604.04
                                          105 Grand Junction, CO                2264.43  439868.47
                                          301 Wilkes-Barre, PA                  2262.27  442130.74
                                           84 El Paso, TX                       2213.93  444344.68
                                          137 Kingston, ON                      2237.67  446582.35
                                            4 Albuquerque, NM                    2170.5  448752.84
                                          272 Syracuse, NY                      2178.26   450931.1
                                          246 Santa Fe, NM                      2120.23  453051.33
                                          217 Reading, PA                       2098.29  455149.62
                                           74 Denver, CO                        2008.13  457157.75
                                          208 Portsmouth, VA                    1992.18  459149.93
                                           57 Cheyenne, WY                      1993.01  461142.94
                                          184 Norfolk, VA                       1993.78  463136.72
                                           62 Colorado Springs, CO              1976.45  465113.17
                                          129 Johnstown, PA                     1972.05  467085.22
                                          212 Pueblo, CO                        1959.85  469045.06
                                          141 Lancaster, PA                     1959.48  471004.54
                                          216 Rapid City, SD                    1881.23  472885.77
                                          303 Wilmington, NC                    1875.13   474760.9
                                          167 Minot, ND                         1881.45  476642.36
                                          275 Tampa, FL                         1912.76  478555.12
                                           36 Brandon, MB                       1940.02  480495.14
                                          101 Gainesville, FL                   1855.41  482350.55
                                           31 Bismarck, ND                      1741.25   484091.8
                                           18 Baltimore, MD                     1749.02  485840.82
                                          143 Laredo, TX                        1779.18  487620.01
                                           24 Belleville, ON                    1918.78  489538.79
                                          156 Lubbock, TX                       1847.76  491386.54
                                          116 Harrisburg, PA                    1786.27  493172.82
                                            7 Amarillo, TX                      1758.66  494931.47
                                          223 Rochester, NY                     1760.64  496692.12
                                           68 Corpus Christi, TX                1730.17  498422.29
                                          279 Timmins, ON                       1812.36  500234.65
                                          238 San Antonio, TX                   1774.49  502009.14
                                          185 North Bay, ON                     1763.41  503772.55
                                            1 Abilene, TX                       1701.18  505473.73
                                          197 Peterborough, ON                  1685.89  507159.62
                                           16 Austin, TX                         1650.4  508810.02
                                          269 Sudbury, ON                       1627.33  510437.35
                                          121 Houston, TX                       1539.52  511976.87
                                           41 Buffalo, NY                       1455.86  513432.73
                                           77 Dodge City, KS                    1503.01  514935.73
                                          294 Washington, DC                    1589.78  516525.51
                                          200 Pierre, SD                        1654.69   518180.2
                                           52 Charleston, SC                    1622.39  519802.59
                                          305 Winnipeg, MB                      1683.01  521485.61
                                          127 Jacksonville, FL                  1729.19   523214.8
                                           92 Fargo, ND                          1549.3   524764.1
                                          220 Richmond, VA                       1482.8  526246.89
                                          292 Waco, TX                          1422.07  527668.96
                                          282 Toronto, ON                       1483.47  529152.43
                                          103 Galveston, TX                     1453.39  530605.82
                                          182 Niagara Falls, ON                 1446.61  532052.43
                                           97 Ft Worth, TX                      1451.32  533503.75
                                          227 Saint Catherines, ON               1447.6  534951.35
                                           69 Dallas, TX                        1414.51  536365.85
                                           42 Burlington, ONT                   1383.45   537749.3
                                          204 Port Arthur, TX                   1349.79   539099.1
                                          111 Guelph, ON                        1334.48  540433.58
                                           23 Beaumont, TX                      1333.91  541767.49
                                          115 Hamilton, ON                      1332.75  543100.24
                                          188 Oklahoma City, OK                 1326.53  544426.77
                                          138 Kitchener, ON                     1306.17  545732.94
                                           85 Enid, OK                          1303.54  547036.48
                                          215 Raleigh, NC                       1330.04  548366.51
                                          258 Sioux Falls, SD                   1358.74  549725.26
                                          250 Savannah, GA                      1337.75  551063.01
                                          228 Saint Cloud, MN                   1296.84  552359.85
                                          274 Tallahassee, FL                   1248.21  553608.06
                                          278 Thunder Bay, ON                   1287.21  554895.27
                                          180 New Orleans, LA                   1273.63  556168.89
                                          249 Sault Ste Marie, ON               1205.19  557374.08
                                           20 Baton Rouge, LA                   1199.28  558573.36
                                           37 Brantford, ON                     1151.44   559724.8
                                          300 Wichita, KS                       1232.93  560957.73
                                           80 Durham, NC                         1279.4  562237.14
                                          236 Salina, KS                         1307.8  563544.94
                                          109 Greensboro, NC                    1245.96   564790.9
                                          257 Sioux City, IA                    1230.47  566021.37
                                           64 Columbia, SC                      1213.26  567234.63
                                          150 Lincoln, NE                       1177.83  568412.46
                                          221 Roanoke, VA                       1181.08  569593.54
                                          189 Omaha, NE                         1139.08  570732.62
                                          306 Winston-Salem, NC                 1141.42  571874.04
                                           79 Duluth, MN                        1103.04  572977.08
                                          195 Pensacola, FL                     1179.96  574157.04
                                          270 Superior, WI                      1175.77  575332.81
                                           14 Augusta, GA                       1152.38  576485.19
                                          166 Minneapolis, MN                    1113.9  577599.09
                                           54 Charlotte, NC                     1091.15  578690.24
                                          233 Saint Paul, MN                    1080.39  579770.63
                                          157 Macon, GA                         1061.49  580832.12
                                           81 Eau Claire, WI                     989.72  581821.84
                                           28 Biloxi, MS                        1012.28  582834.12
                                          152 London, ON                        1016.65  583850.76
                                          160 Marshall, TX                      1158.56  585009.32
                                           86 Erie, PA                          1188.43  586197.75
                                          287 Tulsa, OK                          1174.1  587371.86
                                          201 Pittsburgh, PA                    1144.29  588516.15
                                          256 Shreveport, LA                    1096.48  589612.63
                                          310 Youngstown, OH                    1081.83  590694.46
                                          277 Texarkana, TX                     1066.85  591761.31
                                          267 Steubenville, OH                  1043.65  592804.96
                                          281 Topeka, KS                        1043.45  593848.41
                                          297 Wheeling, WV                      1035.82  594884.23
                                           95 Ft Smith, AR                       998.79  595883.02
                                           47 Canton, OH                         974.26  596857.28
                                          130 Joplin, MO                         943.13  597800.41
                                            2 Akron, OH                          939.34  598739.75
                                          176 Natchez, MS                        948.25  599688.01
                                           22 Bay City, MI                       980.29   600668.3
                                          112 Gulfport, MS                       982.13  601650.43
                                          108 Green Bay, WI                      980.65  602631.08
                                          168 Mobile, AL                         955.25  603586.33
                                          222 Rochester, MN                      970.35  604556.68
                                          260 Spartanburg, NC                    960.77  605517.44
                                          231 Saint Joseph, MO                   952.41  606469.86
                                          110 Greenville, SC                     925.02  607394.87
                                           75 Des Moines, IA                     904.35  608299.23
                                           10 Asheville, NC                      869.08   609168.3
                                          133 Kansas City, KS                     868.8   610037.1
                                           61 Cleveland, OH                      908.59  610945.69
                                          134 Kansas City, MO                    905.45  611851.14
                                           53 Charleston, WV                     895.98  612747.12
                                          264 Springfield, MO                    809.82  613556.95
                                          312 Zanesville, OH                     802.13  614359.08
                                          151 Little Rock, AR                    795.57  615154.65
                                          226 Saginaw, MI                        831.32  615985.97
                                          126 Jackson, MS                         880.9  616866.87
                                           94 Flint, MI                          865.78  617732.65
                                          170 Montgomery, AL                     757.41  618490.06
                                          295 Waterloo, IA                       814.78  619304.84
                                           65 Columbus, GA                       859.51  620164.34
                                           49 Cedar Rapids, IA                   804.18  620968.53
                                           12 Atlanta, GA                        759.63  621728.16
                                           78 Dubuque, IA                        744.13  622472.29
                                           30 Birmingham, AL                     675.42   623147.7
                                          253 Sheboygan, WI                      709.66  623857.37
                                          100 Gadsden, AL                        683.03   624540.4
                                          158 Madison, WI                        668.43  625208.83
                                           56 Chattanooga, TN                    622.55  625831.37
                                          124 Iowa City, IA                      627.43   626458.8
                                          139 Knoxville, TN                      656.94  627115.74
                                           63 Columbia, MO                       616.95  627732.69
                                          304 Windsor, ON                         687.3  628419.98
                                          162 Memphis, TN                        694.56  629114.55
                                           76 Detroit, MI                        693.11  629807.66
                                          122 Huntsville, AL                     579.38  630387.04
                                          165 Milwaukee, WI                      581.29  630968.33
                                           11 Ashland, KY                        481.46  631449.79
                                          232 Saint Louis, MO                    522.45  631972.25
                                            9 Ann Arbor, MI                      513.15   632485.4
                                          192 Paducah, KY                        491.81   632977.2
                                          142 Lansing, MI                        480.06  633457.26
                                          175 Nashville, TN                      479.16  633936.42
                                          106 Grand Rapids, MI                   475.97  634412.39
                                           35 Bowling Green, KY                  416.17  634828.56
                                          224 Rockford, IL                       408.25  635236.82
                                           66 Columbus, OH                       450.37  635687.19
                                          262 Springfield, IL                    459.25  636146.44
                                          280 Toledo, OH                         439.91  636586.35
                                          196 Peoria, IL                         422.25  637008.59
                                          265 Springfield, OH                    402.89  637411.49
                                           32 Bloomington, IL                    360.32  637771.81
                                          147 Lexington, KY                      356.49  638128.31
                                          214 Racine, WI                         399.12  638527.43
                                          154 Louisville, KY                     339.14  638866.57
                                          135 Kenosha, WI                         331.4  639197.97
                                           60 Cincinnati, OH                     331.61  639529.58
                                           73 Decatur, IL                        314.29  639843.88
                                          125 Jackson, MI                        355.83   640199.7
                                           89 Evansville, IN                     366.87  640566.58
                                           21 Battle Creek, MI                   342.27  640908.84
                                           51 Champaign, IL                       260.8  641169.64
                                          149 Lima, OH                           289.17  641458.82
                                          288 Urbana, IL                         286.77  641745.59
                                           71 Dayton, OH                         278.52  642024.11
                                           58 Chicago, IL                        279.24  642303.35
                                          114 Hamilton, OH                       272.42  642575.77
                                          104 Gary, IN                           244.97  642820.74
                                          174 Muncie, IN                         166.43  642987.17
                                          276 Terre Haute, IN                    148.82  643135.98
                                          132 Kalamazoo, MI                      232.44  643368.42
                                          123 Indianapolis, IN                   178.76  643547.18
                                          259 South Bend, IN                     132.47  643679.65
                                          140 Lafayette, IN                       97.61  643777.26
                                           96 Ft Wayne, IN                       130.36  643907.62
                                          183 Nome, AK                          5777.13  649684.75
                      2  649684.62        183 Nome, AK
                                          230 Saint John's, NF                  7870.89    7870.89
                                          148 Lihue, HI                         7576.95   15447.85
                                            5 Alert, NT                         7905.76   23353.61
                                          119 Hilo, HI                          7786.39   31139.99
                                          271 Syndey, NS                        6867.36   38007.35
                                          120 Honolulu, HI                      6973.04   44980.39
                                           55 Charlottetown, PE                 6769.69   51750.08
                                            8 Anchorage, AK                     6084.17   57834.25
                                          243 San Juan, PR                       6499.8   64334.05
                                           90 Fairbanks, AK                     6485.59   70819.64
                                          113 Halifax, NS                        5979.6   76799.24
                                           70 Dawson, YT                        5390.11   82189.35
                                          169 Moncton, NB                       5290.95    87480.3
                                          298 Whitehorse, YK                    4966.85   92447.16
                                          229 Saint John, NB                    4896.82   97343.98
                                          131 Juneau, AK                        4819.35  102163.34
                                           98 Fredericton, NB                   4768.78  106932.11
                                          209 Prince Rupert, BC                 4454.24  111386.35
                                           19 Bangor, ME                        4310.99  115697.34
                                           88 Eureka, CA                        3836.77  119534.11
                                           15 Augusta, ME                        3765.4  123299.51
                                          291 Victoria, BC                      3718.78  127018.29
                                          205 Portland, ME                      3689.93  130708.23
                                          290 Vancouver, BC                      3676.9  134385.13
                                          207 Portsmouth, NH                    3646.55  138031.68
                                           87 Eugene, OR                        3615.89  141647.57
                                           40 Brockton, MA                      3600.15  145247.72
                                          235 Salem, OR                         3599.44  148847.16
                                           34 Boston, MA                        3595.59  152442.75
                                           25 Bellingham, WA                    3580.78  156023.53
                                           91 Fall River, MA                    3580.15  159603.68
                                          206 Portland, OR                      3569.55  163173.24
                                           46 Cambridge, MA                     3569.83  166743.06
                                          273 Tacoma, WA                        3563.14   170306.2
                                          145 Lawrence, MA                      3557.08  173863.27
                                          241 San Francisco, CA                 3557.84  177421.11
                                          213 Quebec City, QC                      3607  181028.11
                                          186 Oakland, CA                       3596.53  184624.65
                                          155 Lowell, MA                        3536.42  188161.06
                                          252 Seattle, WA                       3541.57  191702.63
                                          210 Providence, RI                     3540.8  195243.43
                                           26 Berkeley, CA                      3524.68  198768.11
                                          254 Sherbrooke, QC                    3533.82  202301.93
                                          245 Santa Cruz, CA                    3527.03  205828.96
                                          159 Manchester, NH                    3519.15  209348.11
                                          242 San Jose, CA                      3506.93  212855.04
                                           67 Concord, NH                       3502.92  216357.96
                                          225 Sacramento, CA                    3466.46  219824.42
                                          307 Worcester, MA                     3442.81  223267.23
                                          268 Stockton, CA                      3432.24  226699.47
                                          284 Trois-Rivieres, QC                3408.48  230107.95
                                          244 Santa Barbara, CA                 3351.71  233459.66
                                          171 Montpelier, VT                     3326.1  236785.76
                                          308 Yakima, WA                        3315.62  240101.39
                                          263 Springfield, MA                   3325.25  243426.64
                                           99 Fresno, CA                        3280.93  246707.57
                                           38 Brattleboro, VT                   3289.35  249996.92
                                          219 Reno, NV                          3273.12  253270.05
                                          117 Hartford, CT                      3259.94  256529.98
                                           48 Carson City, NV                   3258.04  259788.02
                                          178 New Britain,CT                    3251.14  263039.16
                                           17 Bakersfield, CA                   3224.24   266263.4
                                           43 Burlington, VT                    3226.83  269490.23
                                          153 Los Angeles, CA                   3193.66  272683.89
                                          163 Meriden, CT                        3181.7  275865.59
                                          309 Yellowknife, NT                   3219.55  279085.14
                                          164 Miami, FL                         3467.82  282552.96
                                           82 Edmonton, AB                      2998.65  285551.61
                                          136 Key West, FL                      2971.34  288522.96
                                          261 Spokane, WA                       2934.85  291457.81
                                          179 New Haven, CT                     3105.67  294563.47
                                          193 Pasadena, CA                      3163.09  297726.57
                                          172 Montreal, QC                      3157.19  300883.76
                                          240 San Diego, CA                     3118.23  304001.99
                                          202 Pittsfield, MA                     3107.7  307109.68
                                          293 Walla Walla, WA                   3125.94  310235.63
                                           50 Central Islip, NY                 3140.18   313375.8
                                          239 San Bernardino, CA                3080.98  316456.79
                                           39 Bridgeport, CT                    3084.78  319541.56
                                           33 Boise, ID                         2975.73  322517.29
                                          266 Stamford, CT                      2952.96  325470.25
                                          311 Yuma, AZ                           2896.3  328366.55
                                          285 Troy, NY                          2911.41  331277.97
                                          144 Las Vegas, NV                      2899.2  334177.16
                                            3 Albany, NY                        2893.98  337071.14
                                           45 Calgary, AB                       2842.81  339913.95
                                          296 West Palm Beach, FL               2889.69  342803.64
                                          146 Lethbridge, AB                    2761.59  345565.23
                                          299 White Plains, NY                  2761.87   348327.1
                                           44 Butte, MT                         2700.79  351027.89
                                          181 New York, NY                      2687.05  353714.95
                                          199 Phoenix, AZ                       2677.74  356392.69
                                          251 Schenectady, NY                   2713.16  359105.85
                                          203 Pocatello, ID                     2660.51  361766.36
                                          128 Jersey City, NJ                    2655.1  364421.46
                                          118 Helena, MT                         2653.8  367075.27
                                          177 Newark, NJ                        2647.25  369722.51
                                           93 Flagstaff, AZ                     2617.66  372340.18
                                          194 Paterson, NJ                      2619.56  374959.74
                                          187 Ogden, UT                         2611.96   377571.7
                                           83 Elizabeth, NJ                     2609.47  380181.16
                                          107 Great Falls, MT                   2605.85  382787.01
                                           13 Atlantic City, NJ                  2609.3  385396.31
                                          161 Medicine Hat, AB                  2613.82  388010.13
                                          283 Trenton, NJ                        2576.5  390586.63
                                          237 Salt Lake City, UT                2566.97  393153.59
                                          289 Utica, NY                         2538.01   395691.6
                                          286 Tucson, AZ                        2578.23  398269.82
                                          191 Ottawa, ON                        2575.89  400845.71
                                          211 Provo, UT                          2487.2  403332.91
                                          198 Philadelphia, PA                  2521.63  405854.54
                                           27 Billings, MT                      2338.33  408192.87
                                          302 Wilmington, DE                    2314.81  410507.68
                                          248 Saskatoon, SK                     2317.41  412825.09
                                          247 Sarasota, FL                      2394.44  415219.54
                                           59 Churchill, MB                     2311.24  417530.77
                                          234 Saint Petersburg, FL              2279.52  419810.29
                                          173 Moose Jaw, SK                     2220.06  422030.35
                                          190 Orlando, FL                       2249.04  424279.39
                                          218 Regina, SK                        2208.47  426487.86
                                           72 Daytona Beach, FL                 2195.17  428683.03
                                          255 Sheridan, WY                      2090.58  430773.61
                                            6 Allentown, PA                     2193.28  432966.89
                                          102 Gallup, NM                        2324.19  435291.08
                                           29 Binghamtom, NY                    2312.96  437604.04
                                          105 Grand Junction, CO                2264.43  439868.47
                                          301 Wilkes-Barre, PA                  2262.27  442130.74
                                           84 El Paso, TX                       2213.93  444344.68
                                          137 Kingston, ON                      2237.67  446582.35
                                            4 Albuquerque, NM                    2170.5  448752.84
                                          272 Syracuse, NY                      2178.26   450931.1
                                          246 Santa Fe, NM                      2120.23  453051.33
                                          217 Reading, PA                       2098.29  455149.62
                                           74 Denver, CO                        2008.13  457157.75
                                          208 Portsmouth, VA                    1992.18  459149.93
                                           57 Cheyenne, WY                      1993.01  461142.94
                                          184 Norfolk, VA                       1993.78  463136.72
                                           62 Colorado Springs, CO              1976.45  465113.17
                                          129 Johnstown, PA                     1972.05  467085.22
                                          212 Pueblo, CO                        1959.85  469045.06
                                          141 Lancaster, PA                     1959.48  471004.54
                                          216 Rapid City, SD                    1881.23  472885.77
                                          303 Wilmington, NC                    1875.13   474760.9
                                          167 Minot, ND                         1881.45  476642.36
                                          275 Tampa, FL                         1912.76  478555.12
                                           36 Brandon, MB                       1940.02  480495.14
                                          101 Gainesville, FL                   1855.41  482350.55
                                           31 Bismarck, ND                      1741.25   484091.8
                                           18 Baltimore, MD                     1749.02  485840.82
                                          143 Laredo, TX                        1779.18  487620.01
                                           24 Belleville, ON                    1918.78  489538.79
                                          156 Lubbock, TX                       1847.76  491386.54
                                          116 Harrisburg, PA                    1786.27  493172.82
                                            7 Amarillo, TX                      1758.66  494931.47
                                          223 Rochester, NY                     1760.64  496692.12
                                           68 Corpus Christi, TX                1730.17  498422.29
                                          279 Timmins, ON                       1812.36  500234.65
                                          238 San Antonio, TX                   1774.49  502009.14
                                          185 North Bay, ON                     1763.41  503772.55
                                            1 Abilene, TX                       1701.18  505473.73
                                          197 Peterborough, ON                  1685.89  507159.62
                                           16 Austin, TX                         1650.4  508810.02
                                          269 Sudbury, ON                       1627.33  510437.35
                                          121 Houston, TX                       1539.52  511976.87
                                           41 Buffalo, NY                       1455.86  513432.73
                                           77 Dodge City, KS                    1503.01  514935.73
                                          294 Washington, DC                    1589.78  516525.51
                                          200 Pierre, SD                        1654.69   518180.2
                                           52 Charleston, SC                    1622.39  519802.59
                                          305 Winnipeg, MB                      1683.01  521485.61
                                          127 Jacksonville, FL                  1729.19   523214.8
                                           92 Fargo, ND                          1549.3   524764.1
                                          220 Richmond, VA                       1482.8  526246.89
                                          292 Waco, TX                          1422.07  527668.96
                                          282 Toronto, ON                       1483.47  529152.43
                                          103 Galveston, TX                     1453.39  530605.82
                                          182 Niagara Falls, ON                 1446.61  532052.43
                                           97 Ft Worth, TX                      1451.32  533503.75
                                          227 Saint Catherines, ON               1447.6  534951.35
                                           69 Dallas, TX                        1414.51  536365.85
                                           42 Burlington, ONT                   1383.45   537749.3
                                           23 Beaumont, TX                      1349.56  539098.86
                                          111 Guelph, ON                        1333.91  540432.77
                                          204 Port Arthur, TX                   1334.48  541767.25
                                          115 Hamilton, ON                      1332.86  543100.11
                                          188 Oklahoma City, OK                 1326.53  544426.64
                                          138 Kitchener, ON                     1306.17  545732.81
                                           85 Enid, OK                          1303.54  547036.35
                                          215 Raleigh, NC                       1330.04  548366.39
                                          258 Sioux Falls, SD                   1358.74  549725.13
                                          250 Savannah, GA                      1337.75  551062.88
                                          228 Saint Cloud, MN                   1296.84  552359.72
                                          274 Tallahassee, FL                   1248.21  553607.93
                                          278 Thunder Bay, ON                   1287.21  554895.14
                                          180 New Orleans, LA                   1273.63  556168.77
                                          249 Sault Ste Marie, ON               1205.19  557373.95
                                           20 Baton Rouge, LA                   1199.28  558573.23
                                           37 Brantford, ON                     1151.44  559724.67
                                          300 Wichita, KS                       1232.93   560957.6
                                           80 Durham, NC                         1279.4  562237.01
                                          236 Salina, KS                         1307.8  563544.81
                                          109 Greensboro, NC                    1245.96  564790.77
                                          257 Sioux City, IA                    1230.47  566021.24
                                           64 Columbia, SC                      1213.26   567234.5
                                          150 Lincoln, NE                       1177.83  568412.33
                                          221 Roanoke, VA                       1181.08  569593.41
                                          189 Omaha, NE                         1139.08  570732.49
                                          306 Winston-Salem, NC                 1141.42  571873.91
                                           79 Duluth, MN                        1103.04  572976.95
                                          195 Pensacola, FL                     1179.96  574156.91
                                          270 Superior, WI                      1175.77  575332.68
                                           14 Augusta, GA                       1152.38  576485.06
                                          166 Minneapolis, MN                    1113.9  577598.96
                                           54 Charlotte, NC                     1091.15  578690.11
                                          233 Saint Paul, MN                    1080.39   579770.5
                                          157 Macon, GA                         1061.49  580831.99
                                           81 Eau Claire, WI                     989.72  581821.71
                                           28 Biloxi, MS                        1012.28  582833.99
                                          152 London, ON                        1016.65  583850.64
                                          160 Marshall, TX                      1158.56  585009.19
                                           86 Erie, PA                          1188.43  586197.63
                                          287 Tulsa, OK                          1174.1  587371.73
                                          201 Pittsburgh, PA                    1144.29  588516.02
                                          256 Shreveport, LA                    1096.48   589612.5
                                          310 Youngstown, OH                    1081.83  590694.33
                                          277 Texarkana, TX                     1066.85  591761.18
                                          267 Steubenville, OH                  1043.65  592804.83
                                          281 Topeka, KS                        1043.45  593848.28
                                          297 Wheeling, WV                      1035.82   594884.1
                                           95 Ft Smith, AR                       998.79  595882.89
                                           47 Canton, OH                         974.26  596857.16
                                          130 Joplin, MO                         943.13  597800.28
                                            2 Akron, OH                          939.34  598739.63
                                          176 Natchez, MS                        948.25  599687.88
                                           22 Bay City, MI                       980.29  600668.17
                                          112 Gulfport, MS                       982.13   601650.3
                                          108 Green Bay, WI                      980.65  602630.95
                                          168 Mobile, AL                         955.25   603586.2
                                          222 Rochester, MN                      970.35  604556.55
                                          260 Spartanburg, NC                    960.77  605517.32
                                          231 Saint Joseph, MO                   952.41  606469.73
                                          110 Greenville, SC                     925.02  607394.74
                                           75 Des Moines, IA                     904.35   608299.1
                                           10 Asheville, NC                      869.08  609168.17
                                          133 Kansas City, KS                     868.8  610036.97
                                           61 Cleveland, OH                      908.59  610945.56
                                          134 Kansas City, MO                    905.45  611851.01
                                           53 Charleston, WV                     895.98  612746.99
                                          264 Springfield, MO                    809.82  613556.82
                                          312 Zanesville, OH                     802.13  614358.95
                                          151 Little Rock, AR                    795.57  615154.52
                                          226 Saginaw, MI                        831.32  615985.85
                                          126 Jackson, MS                         880.9  616866.74
                                           94 Flint, MI                          865.78  617732.52
                                          170 Montgomery, AL                     757.41  618489.93
                                          295 Waterloo, IA                       814.78  619304.71
                                           65 Columbus, GA                       859.51  620164.22
                                           49 Cedar Rapids, IA                   804.18   620968.4
                                           12 Atlanta, GA                        759.63  621728.03
                                           78 Dubuque, IA                        744.13  622472.16
                                           30 Birmingham, AL                     675.42  623147.57
                                          253 Sheboygan, WI                      709.66  623857.24
                                          100 Gadsden, AL                        683.03  624540.27
                                          158 Madison, WI                        668.43   625208.7
                                           56 Chattanooga, TN                    622.55  625831.25
                                          124 Iowa City, IA                      627.43  626458.68
                                          139 Knoxville, TN                      656.94  627115.61
                                           63 Columbia, MO                       616.95  627732.56
                                          304 Windsor, ON                         687.3  628419.86
                                          162 Memphis, TN                        694.56  629114.42
                                           76 Detroit, MI                        693.11  629807.53
                                          122 Huntsville, AL                     579.38  630386.91
                                          165 Milwaukee, WI                      581.29   630968.2
                                           11 Ashland, KY                        481.46  631449.67
                                          232 Saint Louis, MO                    522.45  631972.12
                                            9 Ann Arbor, MI                      513.15  632485.27
                                          192 Paducah, KY                        491.81  632977.08
                                          142 Lansing, MI                        480.06  633457.13
                                          175 Nashville, TN                      479.16  633936.29
                                          106 Grand Rapids, MI                   475.97  634412.26
                                           35 Bowling Green, KY                  416.17  634828.44
                                          224 Rockford, IL                       408.25  635236.69
                                           66 Columbus, OH                       450.37  635687.06
                                          262 Springfield, IL                    459.25  636146.31
                                          280 Toledo, OH                         439.91  636586.22
                                          196 Peoria, IL                         422.25  637008.47
                                          265 Springfield, OH                    402.89  637411.36
                                           32 Bloomington, IL                    360.32  637771.68
                                          147 Lexington, KY                      356.49  638128.18
                                          214 Racine, WI                         399.12   638527.3
                                          154 Louisville, KY                     339.14  638866.44
                                          135 Kenosha, WI                         331.4  639197.84
                                           60 Cincinnati, OH                     331.61  639529.45
                                           73 Decatur, IL                        314.29  639843.75
                                          125 Jackson, MI                        355.83  640199.57
                                           89 Evansville, IN                     366.87  640566.45
                                           21 Battle Creek, MI                   342.27  640908.71
                                           51 Champaign, IL                       260.8  641169.51
                                          149 Lima, OH                           289.17  641458.69
                                          288 Urbana, IL                         286.77  641745.46
                                           71 Dayton, OH                         278.52  642023.98
                                           58 Chicago, IL                        279.24  642303.22
                                          114 Hamilton, OH                       272.42  642575.65
                                          104 Gary, IN                           244.97  642820.61
                                          174 Muncie, IN                         166.43  642987.04
                                          276 Terre Haute, IN                    148.82  643135.86
                                          132 Kalamazoo, MI                      232.44  643368.29
                                          123 Indianapolis, IN                   178.76  643547.05
                                          259 South Bend, IN                     132.47  643679.52
                                          140 Lafayette, IN                       97.61  643777.13
                                           96 Ft Wayne, IN                       130.36  643907.49
                                          183 Nome, AK                          5777.13  649684.62
5 -> 148              1  650614.77          5 Alert, NT
                                          148 Lihue, HI                         7905.76    7905.76
                                          230 Saint John's, NF                  7576.95   15482.71
                                          183 Nome, AK                          7870.89   23353.61
                                          271 Syndey, NS                        7389.33   30742.93
                                          120 Honolulu, HI                      6973.04   37715.97
                                           55 Charlottetown, PE                 6769.69   44485.66
                                          119 Hilo, HI                          6665.33   51150.99
                                          113 Halifax, NS                       6613.34   57764.33
                                            8 Anchorage, AK                     6075.13   63839.46
                                          243 San Juan, PR                       6499.8   70339.26
                                           90 Fairbanks, AK                     6485.59   76824.85
                                          169 Moncton, NB                        5881.4   82706.26
                                           70 Dawson, YT                        5290.95   87997.21
                                          229 Saint John, NB                    5223.21   93220.42
                                          298 Whitehorse, YK                    4896.82   98117.25
                                           98 Fredericton, NB                   4845.34  102962.58
                                          131 Juneau, AK                        4768.78  107731.36
                                           19 Bangor, ME                        4630.48  112361.84
                                          209 Prince Rupert, BC                 4310.99  116672.83
                                           15 Augusta, ME                       4247.95  120920.78
                                           88 Eureka, CA                         3765.4  124686.18
                                          205 Portland, ME                      3729.96  128416.14
                                          291 Victoria, BC                      3689.93  132106.07
                                          207 Portsmouth, NH                     3659.1  135765.18
                                          290 Vancouver, BC                     3646.55  139411.72
                                           40 Brockton, MA                      3637.58  143049.31
                                          235 Salem, OR                         3599.44  146648.74
                                           34 Boston, MA                        3595.59  150244.33
                                           87 Eugene, OR                        3596.62  153840.95
                                           46 Cambridge, MA                     3593.43  157434.38
                                           25 Bellingham, WA                    3577.51  161011.89
                                           91 Fall River, MA                    3580.15  164592.04
                                          206 Portland, OR                      3569.55  168161.59
                                          145 Lawrence, MA                      3564.53  171726.12
                                          241 San Francisco, CA                 3557.84  175283.96
                                          213 Quebec City, QC                      3607  178890.96
                                          186 Oakland, CA                       3596.53  182487.49
                                          155 Lowell, MA                        3536.42  186023.91
                                          273 Tacoma, WA                           3547  189570.91
                                          210 Providence, RI                    3545.83  193116.74
                                          252 Seattle, WA                        3540.8  196657.53
                                          159 Manchester, NH                    3529.69  200187.22
                                           26 Berkeley, CA                      3528.99  203716.21
                                          254 Sherbrooke, QC                    3533.82  207250.03
                                          245 Santa Cruz, CA                    3527.03  210777.06
                                           67 Concord, NH                       3515.24   214292.3
                                          242 San Jose, CA                      3502.92  217795.22
                                          307 Worcester, MA                     3477.76  221272.98
                                          225 Sacramento, CA                    3442.81  224715.79
                                          284 Trois-Rivieres, QC                 3415.3  228131.09
                                          268 Stockton, CA                      3408.48  231539.57
                                          171 Montpelier, VT                    3393.96  234933.53
                                          244 Santa Barbara, CA                  3326.1  238259.63
                                           38 Brattleboro, VT                   3308.76  241568.39
                                          308 Yakima, WA                           3323   244891.4
                                          263 Springfield, MA                   3325.25  248216.65
                                           99 Fresno, CA                        3280.93  251497.58
                                          117 Hartford, CT                      3271.83  254769.41
                                          219 Reno, NV                          3259.94  258029.35
                                          178 New Britain,CT                     3253.1  261282.44
                                           48 Carson City, NV                   3251.14  264533.58
                                          163 Meriden, CT                        3248.8  267782.39
                                           17 Bakersfield, CA                   3221.22  271003.61
                                           43 Burlington, VT                    3226.83  274230.43
                                          153 Los Angeles, CA                   3193.66   277424.1
                                          179 New Haven, CT                      3170.9  280594.99
                                          309 Yellowknife, NT                   3219.25  283814.25
                                          164 Miami, FL                         3467.82  287282.06
                                           82 Edmonton, AB                      2998.65  290280.72
                                          136 Key West, FL                      2971.34  293252.06
                                          261 Spokane, WA                       2934.85  296186.91
                                           50 Central Islip, NY                 3092.23  299279.15
                                          293 Walla Walla, WA                   3140.18  302419.32
                                           39 Bridgeport, CT                    3137.06  305556.38
                                          193 Pasadena, CA                      3142.69  308699.07
                                          172 Montreal, QC                      3157.19  311856.26
                                          240 San Diego, CA                     3118.23  314974.49
                                          202 Pittsfield, MA                     3107.7  318082.19
                                          239 San Bernardino, CA                3097.28  321179.47
                                          285 Troy, NY                          3070.66  324250.12
                                           33 Boise, ID                         2937.92  327188.04
                                          266 Stamford, CT                      2952.96     330141
                                          311 Yuma, AZ                           2896.3   333037.3
                                            3 Albany, NY                        2905.84  335943.14
                                          144 Las Vegas, NV                     2893.98  338837.12
                                          251 Schenectady, NY                   2883.23  341720.35
                                           45 Calgary, AB                       2828.13  344548.48
                                          296 West Palm Beach, FL               2889.69  347438.17
                                          146 Lethbridge, AB                    2761.59  350199.76
                                          299 White Plains, NY                  2761.87  352961.63
                                           44 Butte, MT                         2700.79  355662.42
                                          181 New York, NY                      2687.05  358349.48
                                          199 Phoenix, AZ                       2677.74  361027.22
                                          128 Jersey City, NJ                   2673.06  363700.28
                                          203 Pocatello, ID                      2655.1  366355.39
                                          177 Newark, NJ                        2648.53  369003.92
                                          118 Helena, MT                        2647.25  371651.17
                                           13 Atlantic City, NJ                 2646.38  374297.55
                                          161 Medicine Hat, AB                  2613.82  376911.37
                                           83 Elizabeth, NJ                     2604.16  379515.53
                                           93 Flagstaff, AZ                     2614.32  382129.85
                                          194 Paterson, NJ                      2619.56  384749.41
                                          187 Ogden, UT                         2611.96  387361.37
                                          283 Trenton, NJ                       2573.35  389934.72
                                          107 Great Falls, MT                   2575.55  392510.27
                                          198 Philadelphia, PA                  2550.72  395060.99
                                          237 Salt Lake City, UT                2538.22  397599.21
                                          289 Utica, NY                         2538.01  400137.22
                                          286 Tucson, AZ                        2578.23  402715.44
                                          191 Ottawa, ON                        2575.89  405291.33
                                          211 Provo, UT                          2487.2  407778.53
                                            6 Allentown, PA                     2499.13  410277.67
                                          102 Gallup, NM                        2324.19  412601.86
                                           29 Binghamtom, NY                    2312.96  414914.82
                                           27 Billings, MT                       2265.6  417180.42
                                          302 Wilmington, DE                    2314.81  419495.23
                                          248 Saskatoon, SK                     2317.41  421812.65
                                          247 Sarasota, FL                      2394.44  424207.09
                                           59 Churchill, MB                     2311.24  426518.32
                                          234 Saint Petersburg, FL              2279.52  428797.84
                                          173 Moose Jaw, SK                     2220.06   431017.9
                                          190 Orlando, FL                       2249.04  433266.94
                                          218 Regina, SK                        2208.47  435475.41
                                           72 Daytona Beach, FL                 2195.17  437670.58
                                          255 Sheridan, WY                      2090.58  439761.16
                                          184 Norfolk, VA                       2189.22  441950.37
                                          105 Grand Junction, CO                2234.59  444184.96
                                          301 Wilkes-Barre, PA                  2262.27  446447.23
                                           84 El Paso, TX                       2213.93  448661.17
                                          137 Kingston, ON                      2237.67  450898.84
                                            4 Albuquerque, NM                    2170.5  453069.33
                                          272 Syracuse, NY                      2178.26  455247.59
                                          246 Santa Fe, NM                      2120.23  457367.82
                                          217 Reading, PA                       2098.29  459466.11
                                           74 Denver, CO                        2008.13  461474.24
                                          208 Portsmouth, VA                    1992.18  463466.42
                                           57 Cheyenne, WY                      1993.01  465459.43
                                          141 Lancaster, PA                     1971.66  467431.09
                                           62 Colorado Springs, CO              1972.03  469403.12
                                          129 Johnstown, PA                     1972.05  471375.16
                                          212 Pueblo, CO                        1959.85  473335.01
                                           18 Baltimore, MD                     1935.73  475270.74
                                          216 Rapid City, SD                    1868.74  477139.48
                                          303 Wilmington, NC                    1875.13  479014.61
                                          167 Minot, ND                         1881.45  480896.06
                                          275 Tampa, FL                         1912.76  482808.82
                                           36 Brandon, MB                       1940.02  484748.84
                                          101 Gainesville, FL                   1855.41  486604.25
                                           31 Bismarck, ND                      1741.25  488345.51
                                          127 Jacksonville, FL                  1744.35  490089.86
                                          305 Winnipeg, MB                      1729.19  491819.04
                                           52 Charleston, SC                    1683.01  493502.06
                                          200 Pierre, SD                        1622.39  495124.45
                                          294 Washington, DC                    1654.69  496779.14
                                          156 Lubbock, TX                       1753.74  498532.87
                                           24 Belleville, ON                    1847.76  500380.63
                                          143 Laredo, TX                        1918.78  502299.41
                                          279 Timmins, ON                       1920.31  504219.72
                                           68 Corpus Christi, TX                1812.36  506032.08
                                          185 North Bay, ON                      1786.7  507818.78
                                          238 San Antonio, TX                   1763.41  509582.19
                                          223 Rochester, NY                     1726.57  511308.76
                                            7 Amarillo, TX                      1760.64   513069.4
                                          116 Harrisburg, PA                    1758.66  514828.06
                                            1 Abilene, TX                       1668.54   516496.6
                                          197 Peterborough, ON                  1685.89  518182.49
                                           16 Austin, TX                         1650.4  519832.89
                                          269 Sudbury, ON                       1627.33  521460.22
                                          121 Houston, TX                       1539.52  522999.73
                                           41 Buffalo, NY                       1455.86   524455.6
                                           77 Dodge City, KS                    1503.01   525958.6
                                          220 Richmond, VA                      1558.59  527517.19
                                           92 Fargo, ND                          1482.8  528999.99
                                          250 Savannah, GA                      1489.99  530489.98
                                          258 Sioux Falls, SD                   1337.75  531827.74
                                          215 Raleigh, NC                       1358.74  533186.48
                                           85 Enid, OK                          1330.04  534516.52
                                          182 Niagara Falls, ON                 1380.26  535896.78
                                          292 Waco, TX                          1483.03  537379.81
                                          282 Toronto, ON                       1483.47  538863.29
                                           97 Ft Worth, TX                       1449.6  540312.89
                                          227 Saint Catherines, ON               1447.6  541760.49
                                          103 Galveston, TX                     1444.61  543205.09
                                           42 Burlington, ONT                   1421.73  544626.82
                                           69 Dallas, TX                        1383.45  546010.27
                                          115 Hamilton, ON                      1368.85  547379.12
                                           23 Beaumont, TX                      1332.75  548711.87
                                          111 Guelph, ON                        1333.91  550045.78
                                          204 Port Arthur, TX                   1334.48  551380.26
                                          138 Kitchener, ON                     1324.27  552704.53
                                          188 Oklahoma City, OK                 1306.17   554010.7
                                           37 Brantford, ON                     1299.42  555310.11
                                          300 Wichita, KS                       1232.93  556543.05
                                           80 Durham, NC                         1279.4  557822.45
                                          236 Salina, KS                         1307.8  559130.25
                                          109 Greensboro, NC                    1245.96  560376.22
                                          257 Sioux City, IA                    1230.47  561606.68
                                           64 Columbia, SC                      1213.26  562819.94
                                          228 Saint Cloud, MN                   1208.61  564028.55
                                          274 Tallahassee, FL                   1248.21  565276.76
                                          278 Thunder Bay, ON                   1287.21  566563.97
                                          180 New Orleans, LA                   1273.63   567837.6
                                          249 Sault Ste Marie, ON               1205.19  569042.78
                                           20 Baton Rouge, LA                   1199.28  570242.06
                                           79 Duluth, MN                        1130.44  571372.51
                                          195 Pensacola, FL                     1179.96  572552.47
                                          270 Superior, WI                      1175.77  573728.24
                                           14 Augusta, GA                       1152.38  574880.62
                                          150 Lincoln, NE                       1134.43  576015.05
                                          221 Roanoke, VA                       1181.08  577196.13
                                          189 Omaha, NE                         1139.08  578335.21
                                          306 Winston-Salem, NC                 1141.42  579476.63
                                          166 Minneapolis, MN                   1088.91  580565.53
                                           54 Charlotte, NC                     1091.15  581656.69
                                          233 Saint Paul, MN                    1080.39  582737.08
                                          157 Macon, GA                         1061.49  583798.57
                                           81 Eau Claire, WI                     989.72  584788.28
                                           28 Biloxi, MS                        1012.28  585800.56
                                          152 London, ON                        1016.65  586817.21
                                          160 Marshall, TX                      1158.56  587975.77
                                           86 Erie, PA                          1188.43   589164.2
                                          287 Tulsa, OK                          1174.1   590338.3
                                          201 Pittsburgh, PA                    1144.29  591482.59
                                          256 Shreveport, LA                    1096.48  592579.08
                                          310 Youngstown, OH                    1081.83   593660.9
                                          277 Texarkana, TX                     1066.85  594727.75
                                          267 Steubenville, OH                  1043.65  595771.41
                                          281 Topeka, KS                        1043.45  596814.86
                                          297 Wheeling, WV                      1035.82  597850.68
                                           95 Ft Smith, AR                       998.79  598849.47
                                           61 Cleveland, OH                      974.07  599823.54
                                          176 Natchez, MS                        959.96  600783.49
                                           22 Bay City, MI                       980.29  601763.78
                                          112 Gulfport, MS                       982.13  602745.92
                                          108 Green Bay, WI                      980.65  603726.56
                                          168 Mobile, AL                         955.25  604681.82
                                          222 Rochester, MN                      970.35  605652.16
                                          260 Spartanburg, NC                    960.77  606612.93
                                          231 Saint Joseph, MO                   952.41  607565.34
                                           47 Canton, OH                         933.28  608498.62
                                          130 Joplin, MO                         943.13  609441.75
                                            2 Akron, OH                          939.34  610381.09
                                          133 Kansas City, KS                    915.85  611296.95
                                           53 Charleston, WV                     899.41  612196.36
                                          134 Kansas City, MO                    895.98  613092.34
                                          110 Greenville, SC                     891.55  613983.89
                                           75 Des Moines, IA                     904.35  614888.24
                                           10 Asheville, NC                      869.08  615757.32
                                          295 Waterloo, IA                       827.18   616584.5
                                           65 Columbus, GA                       859.51  617444.01
                                          253 Sheboygan, WI                      802.52  618246.52
                                          126 Jackson, MS                         809.5  619056.02
                                          226 Saginaw, MI                         880.9  619936.92
                                          151 Little Rock, AR                    831.32  620768.24
                                          304 Windsor, ON                        828.42  621596.66
                                          264 Springfield, MO                    795.38  622392.04
                                          312 Zanesville, OH                     802.13  623194.17
                                           63 Columbia, MO                       716.36  623910.53
                                           76 Detroit, MI                        682.93  624593.46
                                          170 Montgomery, AL                      724.3  625317.77
                                          158 Madison, WI                        770.17  626087.93
                                           12 Atlanta, GA                        731.47  626819.41
                                           49 Cedar Rapids, IA                   759.63  627579.04
                                          100 Gadsden, AL                        675.88  628254.92
                                           78 Dubuque, IA                        668.88   628923.8
                                           30 Birmingham, AL                     675.42  629599.22
                                           94 Flint, MI                          690.27  630289.49
                                          162 Memphis, TN                        698.85  630988.33
                                          142 Lansing, MI                           647  631635.33
                                          122 Huntsville, AL                     570.44  632205.78
                                          124 Iowa City, IA                      588.25  632794.02
                                          139 Knoxville, TN                      656.94  633450.96
                                          224 Rockford, IL                       563.81  634014.76
                                           56 Chattanooga, TN                    563.58  634578.34
                                          165 Milwaukee, WI                      580.71  635159.05
                                           11 Ashland, KY                        481.46  635640.51
                                          232 Saint Louis, MO                    522.45  636162.96
                                            9 Ann Arbor, MI                      513.15  636676.12
                                          192 Paducah, KY                        491.81  637167.92
                                          125 Jackson, MI                        459.78   637627.7
                                          175 Nashville, TN                      451.22  638078.92
                                          106 Grand Rapids, MI                   475.97  638554.89
                                           35 Bowling Green, KY                  416.17  638971.07
                                          214 Racine, WI                         406.98  639378.05
                                          147 Lexington, KY                      399.12  639777.17
                                          196 Peoria, IL                         399.56  640176.73
                                           66 Columbus, OH                       458.14  640634.87
                                          262 Springfield, IL                    459.25  641094.13
                                          280 Toledo, OH                         439.91  641534.03
                                           73 Decatur, IL                        393.78  641927.82
                                          265 Springfield, OH                    355.61  642283.42
                                           32 Bloomington, IL                    360.32  642643.75
                                          149 Lima, OH                           338.23  642981.98
                                           89 Evansville, IN                     305.64  643287.62
                                           21 Battle Creek, MI                   342.27  643629.89
                                          154 Louisville, KY                     283.85  643913.74
                                          135 Kenosha, WI                         331.4  644245.14
                                           60 Cincinnati, OH                     331.61  644576.75
                                           58 Chicago, IL                         288.4  644865.14
                                           71 Dayton, OH                         279.24  645144.39
                                           51 Champaign, IL                      281.04  645425.42
                                          114 Hamilton, OH                       259.18   645684.6
                                          288 Urbana, IL                         256.66  645941.26
                                          132 Kalamazoo, MI                      235.55  646176.81
                                          276 Terre Haute, IN                    232.44  646409.25
                                           96 Ft Wayne, IN                        195.3  646604.55
                                          104 Gary, IN                           156.52  646761.07
                                          174 Muncie, IN                         166.43   646927.5
                                          259 South Bend, IN                     118.99  647046.49
                                          123 Indianapolis, IN                   132.47  647178.96
                                          140 Lafayette, IN                       66.81  647245.77
                                            5 Alert, NT                         3369.01  650614.77
                      2  650613.92          5 Alert, NT
                                          148 Lihue, HI                         7905.76    7905.76
                                          230 Saint John's, NF                  7576.95   15482.71
                                          183 Nome, AK                          7870.89   23353.61
                                          271 Syndey, NS                        7389.33   30742.93
                                          120 Honolulu, HI                      6973.04   37715.97
                                           55 Charlottetown, PE                 6769.69   44485.66
                                          119 Hilo, HI                          6665.33   51150.99
                                          113 Halifax, NS                       6613.34   57764.33
                                            8 Anchorage, AK                     6075.13   63839.46
                                          243 San Juan, PR                       6499.8   70339.26
                                           90 Fairbanks, AK                     6485.59   76824.85
                                          169 Moncton, NB                        5881.4   82706.26
                                           70 Dawson, YT                        5290.95   87997.21
                                          229 Saint John, NB                    5223.21   93220.42
                                          298 Whitehorse, YK                    4896.82   98117.25
                                           98 Fredericton, NB                   4845.34  102962.58
                                          131 Juneau, AK                        4768.78  107731.36
                                           19 Bangor, ME                        4630.48  112361.84
                                          209 Prince Rupert, BC                 4310.99  116672.83
                                           15 Augusta, ME                       4247.95  120920.78
                                           88 Eureka, CA                         3765.4  124686.18
                                          205 Portland, ME                      3729.96  128416.14
                                          291 Victoria, BC                      3689.93  132106.07
                                          207 Portsmouth, NH                     3659.1  135765.18
                                          290 Vancouver, BC                     3646.55  139411.72
                                           40 Brockton, MA                      3637.58  143049.31
                                          235 Salem, OR                         3599.44  146648.74
                                           34 Boston, MA                        3595.59  150244.33
                                           87 Eugene, OR                        3596.62  153840.95
                                           46 Cambridge, MA                     3593.43  157434.38
                                           25 Bellingham, WA                    3577.51  161011.89
                                           91 Fall River, MA                    3580.15  164592.04
                                          206 Portland, OR                      3569.55  168161.59
                                          145 Lawrence, MA                      3564.53  171726.12
                                          241 San Francisco, CA                 3557.84  175283.96
                                          213 Quebec City, QC                      3607  178890.96
                                          186 Oakland, CA                       3596.53  182487.49
                                          155 Lowell, MA                        3536.42  186023.91
                                          273 Tacoma, WA                           3547  189570.91
                                          210 Providence, RI                    3545.83  193116.74
                                          252 Seattle, WA                        3540.8  196657.53
                                          159 Manchester, NH                    3529.69  200187.22
                                           26 Berkeley, CA                      3528.99  203716.21
                                          254 Sherbrooke, QC                    3533.82  207250.03
                                          245 Santa Cruz, CA                    3527.03  210777.06
                                           67 Concord, NH                       3515.24   214292.3
                                          242 San Jose, CA                      3502.92  217795.22
                                          307 Worcester, MA                     3477.76  221272.98
                                          225 Sacramento, CA                    3442.81  224715.79
                                          284 Trois-Rivieres, QC                 3415.3  228131.09
                                          268 Stockton, CA                      3408.48  231539.57
                                          171 Montpelier, VT                    3393.96  234933.53
                                          244 Santa Barbara, CA                  3326.1  238259.63
                                           38 Brattleboro, VT                   3308.76  241568.39
                                          308 Yakima, WA                           3323   244891.4
                                          263 Springfield, MA                   3325.25  248216.65
                                           99 Fresno, CA                        3280.93  251497.58
                                          117 Hartford, CT                      3271.83  254769.41
                                          219 Reno, NV                          3259.94  258029.35
                                          178 New Britain,CT                     3253.1  261282.44
                                           48 Carson City, NV                   3251.14  264533.58
                                          163 Meriden, CT                        3248.8  267782.39
                                           17 Bakersfield, CA                   3221.22  271003.61
                                           43 Burlington, VT                    3226.83  274230.43
                                          153 Los Angeles, CA                   3193.66   277424.1
                                          179 New Haven, CT                      3170.9  280594.99
                                          309 Yellowknife, NT                   3219.25  283814.25
                                          164 Miami, FL                         3467.82  287282.06
                                           82 Edmonton, AB                      2998.65  290280.72
                                          136 Key West, FL                      2971.34  293252.06
                                          261 Spokane, WA                       2934.85  296186.91
                                           50 Central Islip, NY                 3092.23  299279.15
                                          293 Walla Walla, WA                   3140.18  302419.32
                                           39 Bridgeport, CT                    3137.06  305556.38
                                          193 Pasadena, CA                      3142.69  308699.07
                                          172 Montreal, QC                      3157.19  311856.26
                                          240 San Diego, CA                     3118.23  314974.49
                                          202 Pittsfield, MA                     3107.7  318082.19
                                          239 San Bernardino, CA                3097.28  321179.47
                                          285 Troy, NY                          3070.66  324250.12
                                           33 Boise, ID                         2937.92  327188.04
                                          266 Stamford, CT                      2952.96     330141
                                          311 Yuma, AZ                           2896.3   333037.3
                                            3 Albany, NY                        2905.84  335943.14
                                          144 Las Vegas, NV                     2893.98  338837.12
                                          251 Schenectady, NY                   2883.23  341720.35
                                           45 Calgary, AB                       2828.13  344548.48
                                          296 West Palm Beach, FL               2889.69  347438.17
                                          146 Lethbridge, AB                    2761.59  350199.76
                                          299 White Plains, NY                  2761.87  352961.63
                                           44 Butte, MT                         2700.79  355662.42
                                          181 New York, NY                      2687.05  358349.48
                                          199 Phoenix, AZ                       2677.74  361027.22
                                          128 Jersey City, NJ                   2673.06  363700.28
                                          203 Pocatello, ID                      2655.1  366355.39
                                          177 Newark, NJ                        2648.53  369003.92
                                          118 Helena, MT                        2647.25  371651.17
                                           13 Atlantic City, NJ                 2646.38  374297.55
                                          161 Medicine Hat, AB                  2613.82  376911.37
                                           83 Elizabeth, NJ                     2604.16  379515.53
                                           93 Flagstaff, AZ                     2614.32  382129.85
                                          194 Paterson, NJ                      2619.56  384749.41
                                          187 Ogden, UT                         2611.96  387361.37
                                          283 Trenton, NJ                       2573.35  389934.72
                                          107 Great Falls, MT                   2575.55  392510.27
                                          198 Philadelphia, PA                  2550.72  395060.99
                                          237 Salt Lake City, UT                2538.22  397599.21
                                          289 Utica, NY                         2538.01  400137.22
                                          286 Tucson, AZ                        2578.23  402715.44
                                          191 Ottawa, ON                        2575.89  405291.33
                                          211 Provo, UT                          2487.2  407778.53
                                            6 Allentown, PA                     2499.13  410277.67
                                          102 Gallup, NM                        2324.19  412601.86
                                           29 Binghamtom, NY                    2312.96  414914.82
                                           27 Billings, MT                       2265.6  417180.42
                                          302 Wilmington, DE                    2314.81  419495.23
                                          248 Saskatoon, SK                     2317.41  421812.65
                                          247 Sarasota, FL                      2394.44  424207.09
                                           59 Churchill, MB                     2311.24  426518.32
                                          234 Saint Petersburg, FL              2279.52  428797.84
                                          173 Moose Jaw, SK                     2220.06   431017.9
                                          190 Orlando, FL                       2249.04  433266.94
                                          218 Regina, SK                        2208.47  435475.41
                                           72 Daytona Beach, FL                 2195.17  437670.58
                                          255 Sheridan, WY                      2090.58  439761.16
                                          184 Norfolk, VA                       2189.22  441950.37
                                          105 Grand Junction, CO                2234.59  444184.96
                                          301 Wilkes-Barre, PA                  2262.27  446447.23
                                           84 El Paso, TX                       2213.93  448661.17
                                          137 Kingston, ON                      2237.67  450898.84
                                            4 Albuquerque, NM                    2170.5  453069.33
                                          272 Syracuse, NY                      2178.26  455247.59
                                          246 Santa Fe, NM                      2120.23  457367.82
                                          217 Reading, PA                       2098.29  459466.11
                                           74 Denver, CO                        2008.13  461474.24
                                          208 Portsmouth, VA                    1992.18  463466.42
                                           57 Cheyenne, WY                      1993.01  465459.43
                                          141 Lancaster, PA                     1971.66  467431.09
                                           62 Colorado Springs, CO              1972.03  469403.12
                                          129 Johnstown, PA                     1972.05  471375.16
                                          212 Pueblo, CO                        1959.85  473335.01
                                           18 Baltimore, MD                     1935.73  475270.74
                                          216 Rapid City, SD                    1868.74  477139.48
                                          303 Wilmington, NC                    1875.13  479014.61
                                          167 Minot, ND                         1881.45  480896.06
                                          275 Tampa, FL                         1912.76  482808.82
                                           36 Brandon, MB                       1940.02  484748.84
                                          101 Gainesville, FL                   1855.41  486604.25
                                           31 Bismarck, ND                      1741.25  488345.51
                                          127 Jacksonville, FL                  1744.35  490089.86
                                          305 Winnipeg, MB                      1729.19  491819.04
                                           52 Charleston, SC                    1683.01  493502.06
                                          200 Pierre, SD                        1622.39  495124.45
                                          294 Washington, DC                    1654.69  496779.14
                                          156 Lubbock, TX                       1753.74  498532.87
                                           24 Belleville, ON                    1847.76  500380.63
                                          143 Laredo, TX                        1918.78  502299.41
                                          279 Timmins, ON                       1920.31  504219.72
                                           68 Corpus Christi, TX                1812.36  506032.08
                                          185 North Bay, ON                      1786.7  507818.78
                                          238 San Antonio, TX                   1763.41  509582.19
                                          223 Rochester, NY                     1726.57  511308.76
                                            7 Amarillo, TX                      1760.64   513069.4
                                          116 Harrisburg, PA                    1758.66  514828.06
                                            1 Abilene, TX                       1668.54   516496.6
                                          197 Peterborough, ON                  1685.89  518182.49
                                           16 Austin, TX                         1650.4  519832.89
                                          269 Sudbury, ON                       1627.33  521460.22
                                          121 Houston, TX                       1539.52  522999.73
                                           41 Buffalo, NY                       1455.86   524455.6
                                           77 Dodge City, KS                    1503.01   525958.6
                                          220 Richmond, VA                      1558.59  527517.19
                                           92 Fargo, ND                          1482.8  528999.99
                                          250 Savannah, GA                      1489.99  530489.98
                                          258 Sioux Falls, SD                   1337.75  531827.74
                                          215 Raleigh, NC                       1358.74  533186.48
                                           85 Enid, OK                          1330.04  534516.52
                                          182 Niagara Falls, ON                 1380.26  535896.78
                                          292 Waco, TX                          1483.03  537379.81
                                          282 Toronto, ON                       1483.47  538863.29
                                           97 Ft Worth, TX                       1449.6  540312.89
                                          227 Saint Catherines, ON               1447.6  541760.49
                                          103 Galveston, TX                     1444.61  543205.09
                                           42 Burlington, ONT                   1421.73  544626.82
                                           69 Dallas, TX                        1383.45  546010.27
                                          115 Hamilton, ON                      1368.85  547379.12
                                           23 Beaumont, TX                      1332.75  548711.87
                                          111 Guelph, ON                        1333.91  550045.78
                                          204 Port Arthur, TX                   1334.48  551380.26
                                          138 Kitchener, ON                     1324.27  552704.53
                                          188 Oklahoma City, OK                 1306.17   554010.7
                                           37 Brantford, ON                     1299.42  555310.11
                                          300 Wichita, KS                       1232.93  556543.05
                                           80 Durham, NC                         1279.4  557822.45
                                          236 Salina, KS                         1307.8  559130.25
                                          109 Greensboro, NC                    1245.96  560376.22
                                          257 Sioux City, IA                    1230.47  561606.68
                                           64 Columbia, SC                      1213.26  562819.94
                                          228 Saint Cloud, MN                   1208.61  564028.55
                                          274 Tallahassee, FL                   1248.21  565276.76
                                          278 Thunder Bay, ON                   1287.21  566563.97
                                          180 New Orleans, LA                   1273.63   567837.6
                                          249 Sault Ste Marie, ON               1205.19  569042.78
                                           20 Baton Rouge, LA                   1199.28  570242.06
                                           79 Duluth, MN                        1130.44  571372.51
                                          195 Pensacola, FL                     1179.96  572552.47
                                          270 Superior, WI                      1175.77  573728.24
                                           14 Augusta, GA                       1152.38  574880.62
                                          150 Lincoln, NE                       1134.43  576015.05
                                          221 Roanoke, VA                       1181.08  577196.13
                                          189 Omaha, NE                         1139.08  578335.21
                                          306 Winston-Salem, NC                 1141.42  579476.63
                                          166 Minneapolis, MN                   1088.91  580565.53
                                           54 Charlotte, NC                     1091.15  581656.69
                                          233 Saint Paul, MN                    1080.39  582737.08
                                          157 Macon, GA                         1061.49  583798.57
                                           81 Eau Claire, WI                     989.72  584788.28
                                           28 Biloxi, MS                        1012.28  585800.56
                                          152 London, ON                        1016.65  586817.21
                                          160 Marshall, TX                      1158.56  587975.77
                                           86 Erie, PA                          1188.43   589164.2
                                          287 Tulsa, OK                          1174.1   590338.3
                                          201 Pittsburgh, PA                    1144.29  591482.59
                                          256 Shreveport, LA                    1096.48  592579.08
                                          310 Youngstown, OH                    1081.83   593660.9
                                          277 Texarkana, TX                     1066.85  594727.75
                                          267 Steubenville, OH                  1043.65  595771.41
                                          281 Topeka, KS                        1043.45  596814.86
                                          297 Wheeling, WV                      1035.82  597850.68
                                           95 Ft Smith, AR                       998.79  598849.47
                                           61 Cleveland, OH                      974.07  599823.54
                                          176 Natchez, MS                        959.96  600783.49
                                           22 Bay City, MI                       980.29  601763.78
                                          112 Gulfport, MS                       982.13  602745.92
                                          108 Green Bay, WI                      980.65  603726.56
                                          168 Mobile, AL                         955.25  604681.82
                                          222 Rochester, MN                      970.35  605652.16
                                          260 Spartanburg, NC                    960.77  606612.93
                                          231 Saint Joseph, MO                   952.41  607565.34
                                           47 Canton, OH                         933.28  608498.62
                                          130 Joplin, MO                         943.13  609441.75
                                            2 Akron, OH                          939.34  610381.09
                                          133 Kansas City, KS                    915.85  611296.95
                                           53 Charleston, WV                     899.41  612196.36
                                          134 Kansas City, MO                    895.98  613092.34
                                          110 Greenville, SC                     891.55  613983.89
                                           75 Des Moines, IA                     904.35  614888.24
                                           10 Asheville, NC                      869.08  615757.32
                                          295 Waterloo, IA                       827.18   616584.5
                                           65 Columbus, GA                       859.51  617444.01
                                          253 Sheboygan, WI                      802.52  618246.52
                                          126 Jackson, MS                         809.5  619056.02
                                          226 Saginaw, MI                         880.9  619936.92
                                          151 Little Rock, AR                    831.32  620768.24
                                          304 Windsor, ON                        828.42  621596.66
                                          264 Springfield, MO                    795.38  622392.04
                                          312 Zanesville, OH                     802.13  623194.17
                                           63 Columbia, MO                       716.36  623910.53
                                           76 Detroit, MI                        682.93  624593.46
                                          170 Montgomery, AL                      724.3  625317.77
                                          158 Madison, WI                        770.17  626087.93
                                           12 Atlanta, GA                        731.47  626819.41
                                           49 Cedar Rapids, IA                   759.63  627579.04
                                          100 Gadsden, AL                        675.88  628254.92
                                           78 Dubuque, IA                        668.88   628923.8
                                           30 Birmingham, AL                     675.42  629599.22
                                           94 Flint, MI                          690.27  630289.49
                                          162 Memphis, TN                        698.85  630988.33
                                          142 Lansing, MI                           647  631635.33
                                          122 Huntsville, AL                     570.44  632205.78
                                          124 Iowa City, IA                      588.25  632794.02
                                          139 Knoxville, TN                      656.94  633450.96
                                          224 Rockford, IL                       563.81  634014.76
                                           56 Chattanooga, TN                    563.58  634578.34
                                          165 Milwaukee, WI                      580.71  635159.05
                                           11 Ashland, KY                        481.46  635640.51
                                          232 Saint Louis, MO                    522.45  636162.96
                                            9 Ann Arbor, MI                      513.15  636676.12
                                          192 Paducah, KY                        491.81  637167.92
                                          125 Jackson, MI                        459.78   637627.7
                                          175 Nashville, TN                      451.22  638078.92
                                          106 Grand Rapids, MI                   475.97  638554.89
                                           35 Bowling Green, KY                  416.17  638971.07
                                          214 Racine, WI                         406.98  639378.05
                                          147 Lexington, KY                      399.12  639777.17
                                          196 Peoria, IL                         399.56  640176.73
                                           66 Columbus, OH                       458.14  640634.87
                                          262 Springfield, IL                    459.25  641094.13
                                          280 Toledo, OH                         439.91  641534.03
                                           73 Decatur, IL                        393.78  641927.82
                                          265 Springfield, OH                    355.61  642283.42
                                           32 Bloomington, IL                    360.32  642643.75
                                          149 Lima, OH                           338.23  642981.98
                                           89 Evansville, IN                     305.64  643287.62
                                           21 Battle Creek, MI                   342.27  643629.89
                                          154 Louisville, KY                     283.85  643913.74
                                          135 Kenosha, WI                         331.4  644245.14
                                           60 Cincinnati, OH                     331.61  644576.75
                                           58 Chicago, IL                         288.4  644865.14
                                           71 Dayton, OH                         279.24  645144.39
                                          288 Urbana, IL                         278.52  645422.91
                                          114 Hamilton, OH                       256.66  645679.57
                                           51 Champaign, IL                      259.18  645938.75
                                          132 Kalamazoo, MI                      237.21  646175.96
                                          276 Terre Haute, IN                    232.44   646408.4
                                           96 Ft Wayne, IN                        195.3   646603.7
                                          104 Gary, IN                           156.52  646760.22
                                          174 Muncie, IN                         166.43  646926.65
                                          259 South Bend, IN                     118.99  647045.64
                                          123 Indianapolis, IN                   132.47  647178.11
                                          140 Lafayette, IN                       66.81  647244.92
                                            5 Alert, NT                         3369.01  650613.92

1252 rows selected.

Elapsed: 00:01:24.28

Summary of Results

Here we list the best solutions found for each root leg, i.e. those with PATH_RNK = 1. We do not know what the optima are.

Shortest Routes
===============
ROOT_LEG       TOT_DIST
------------ ----------
184 -> 208     55744.86
51 -> 288      56794.75

Longest Routes
==============
ROOT_LEG       TOT_DIST
------------ ----------
183 -> 230    649684.75
5 -> 148      650614.77

Execution Plan for Shortest Routes, USCA312

----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| Id  | Operation                                             | Name        | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  | Writes |  OMem |  1Mem | Used-Mem |
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT                                      |             |      1 |        |   1252 |00:01:24.88 |      10M|    113K|  10654 |       |       |          |
|   1 |  WINDOW SORT                                          |             |      1 |     10 |   1252 |00:01:24.88 |      10M|    113K|  10654 |   267K|   267K|  237K (0)|
|   2 |   NESTED LOOPS OUTER                                  |             |      1 |     10 |   1252 |00:01:24.88 |      10M|    113K|  10654 |       |       |          |
|   3 |    MERGE JOIN                                         |             |      1 |     10 |   1252 |00:01:24.87 |      10M|    113K|  10654 |       |       |          |
|   4 |     TABLE ACCESS BY INDEX ROWID                       | TOWNS       |      1 |      5 |    312 |00:00:00.01 |     205 |      0 |      0 |       |       |          |
|   5 |      INDEX FULL SCAN                                  | SYS_C008004 |      1 |      5 |    312 |00:00:00.01 |       1 |      0 |      0 |       |       |          |
|*  6 |     SORT JOIN                                         |             |    312 |     10 |   1252 |00:01:24.87 |      10M|    113K|  10654 |   133K|   133K|  118K (0)|
|   7 |      VIEW                                             |             |      1 |     10 |   1252 |00:01:24.86 |      10M|    113K|  10654 |       |       |          |
|   8 |       WINDOW SORT                                     |             |      1 |     10 |   1252 |00:01:24.86 |      10M|    113K|  10654 |  1895K|   658K| 1684K (0)|
|   9 |        MERGE JOIN CARTESIAN                           |             |      1 |     10 |   1252 |00:01:24.85 |      10M|    113K|  10654 |       |       |          |
|  10 |         VIEW                                          |             |      1 |      1 |    313 |00:00:00.01 |       1 |      0 |      0 |       |       |          |
|  11 |          CONNECT BY WITHOUT FILTERING                 |             |      1 |        |    313 |00:00:00.01 |       1 |      0 |      0 |       |       |          |
|  12 |           VIEW                                        |             |      1 |      1 |      1 |00:00:00.01 |       1 |      0 |      0 |       |       |          |
|  13 |            SORT AGGREGATE                             |             |      1 |      1 |      1 |00:00:00.01 |       1 |      0 |      0 |       |       |          |
|  14 |             INDEX FULL SCAN                           | SYS_C008004 |      1 |      5 |    312 |00:00:00.01 |       1 |      0 |      0 |       |       |          |
|  15 |         BUFFER SORT                                   |             |    313 |     10 |   1252 |00:01:24.85 |      10M|    113K|  10654 |  9216 |  9216 | 8192  (0)|
|* 16 |          VIEW                                         |             |      1 |     10 |      4 |00:01:24.85 |      10M|    113K|  10654 |       |       |          |
|* 17 |           WINDOW SORT PUSHED RANK                     |             |      1 |     10 |      4 |00:01:24.85 |      10M|    113K|  10654 |  9216 |  9216 | 8192  (0)|
|  18 |            NESTED LOOPS                               |             |      1 |        |      4 |00:00:34.80 |      10M|    113K|  10654 |       |       |          |
|  19 |             NESTED LOOPS                              |             |      1 |     10 |      4 |00:00:34.80 |      10M|    113K|  10654 |       |       |          |
|* 20 |              VIEW                                     |             |      1 |     12 |      4 |00:00:34.80 |      10M|    113K|  10654 |       |       |          |
|  21 |               UNION ALL (RECURSIVE WITH) BREADTH FIRST|             |      1 |        |    192K|00:00:04.53 |      10M|    113K|  10654 |       |       |          |
|* 22 |                VIEW                                   |             |      1 |     10 |      2 |00:00:00.10 |     693 |      1 |      0 |       |       |          |
|* 23 |                 WINDOW SORT PUSHED RANK               |             |      1 |     10 |      3 |00:00:00.10 |     693 |      1 |      0 |  2048 |  2048 | 2048  (0)|
|  24 |                  WINDOW BUFFER                        |             |      1 |     10 |  48516 |00:00:00.08 |     693 |      1 |      0 |  3100K|   779K| 2755K (0)|
|  25 |                   TABLE ACCESS BY INDEX ROWID         | DISTANCES   |      1 |     10 |  48516 |00:00:00.04 |     693 |      1 |      0 |       |       |          |
|* 26 |                    INDEX FULL SCAN                    | DISTANCE_PK |      1 |     10 |  48516 |00:00:00.02 |     212 |      0 |      0 |       |       |          |
|  27 |                WINDOW SORT                            |             |    311 |      2 |    192K|00:00:04.40 |    5664 |      1 |      0 |   619K|   472K|  550K (0)|
|  28 |                 NESTED LOOPS                          |             |    311 |        |    192K|00:00:05.39 |    5664 |      1 |      0 |       |       |          |
|  29 |                  NESTED LOOPS                         |             |    311 |      2 |    192K|00:00:05.08 |    2392 |      0 |      0 |       |       |          |
|  30 |                   RECURSIVE WITH PUMP                 |             |    311 |        |   1242 |00:00:00.01 |       0 |      0 |      0 |       |       |          |
|* 31 |                   INDEX RANGE SCAN                    | DISTANCE_PK |   1242 |      1 |    192K|00:00:06.49 |    2392 |      0 |      0 |       |       |          |
|  32 |                  TABLE ACCESS BY INDEX ROWID          | DISTANCES   |    192K|      1 |    192K|00:00:00.15 |    3272 |      1 |      0 |       |       |          |
|* 33 |              INDEX UNIQUE SCAN                        | DISTANCE_PK |      4 |      1 |      4 |00:00:00.01 |       6 |      0 |      0 |       |       |          |
|  34 |             TABLE ACCESS BY INDEX ROWID               | DISTANCES   |      4 |      1 |      4 |00:00:00.01 |       4 |      0 |      0 |       |       |          |
|  35 |    TABLE ACCESS BY INDEX ROWID                        | DISTANCES   |   1252 |      1 |   1248 |00:00:00.01 |    2498 |      0 |      0 |       |       |          |
|* 36 |     INDEX UNIQUE SCAN                                 | DISTANCE_PK |   1252 |      1 |   1248 |00:00:00.01 |    1250 |      0 |      0 |       |       |          |
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   6 - access("TWN"."ID"="TOP"."TOWN_ID")
       filter("TWN"."ID"="TOP"."TOWN_ID")
  16 - filter("CIRCUITS"."PATH_RNK"<=:KEEP_NUM)
  17 - filter(ROW_NUMBER() OVER ( PARTITION BY "R"."ROOT_LEG" ORDER BY :SIGN*("R"."TOT_PRICE"+"D"."DST"))<=:KEEP_NUM)
  20 - filter(("R"."LEV"="R"."N_TOWNS" AND "R"."PATH_RNK"<=:KEEP_NUM))
  22 - filter("D"."RNK_BY_DST"<=:KEEP_NUM_ROOT)
  23 - filter(ROW_NUMBER() OVER ( ORDER BY :SIGN*"DST")<=:KEEP_NUM_ROOT) 26 - filter("B">"A")
  31 - access("D"."A"="R"."NXT_ID")
       filter("R"."PATH" NOT LIKE '%|'||LPAD(TO_CHAR("D"."B"),3,'0')||'%')
  33 - access("D"."A"="R"."NXT_ID" AND "D"."B"="R"."ROOT")
  36 - access("DST"."A"="TOP"."TOWN_ID_PRIOR" AND "DST"."B"="TOP"."TOWN_ID")

Notes

  • In the closed version of TSP that we have considered here, the starting point does not essentially affect the set of possible solutions; however, it may affect the solutions actually obtained by an approximate algorithm; partitioning by a set of root legs is designed to improve the quality of solutions obtained
  • I found that the query uses increasing amounts of temp tablespace as the keep values are increased for the larger problem; in my previous article, on a knapsack problem, I included a PL/SQL solution that had the flexibility to discard all but the best solutions as it went, thus keeping space usage to a minimum; it also had the flexibility of the conventional 'branch and bound' algorithms to retain the best solution values globally and close off paths that could not beat them. This is difficult to do in SQL, but against that, the SQL is much simpler

Conclusions

Following my earlier two articles on related subjects, we have again seen that the recursive capabilities of Oracle's SQL from v11.2 can provide surprisingly simple approximate solutions to 'hard' combinatorial problems in reasonable execution times.

20 February 2016: Added attachment with the input data files, DDL and raw results:

TSP Files: input data files, DDL and raw results






SQL for the Fantasy Football Knapsack Problem

Someone (FilippeSoaresRoza) asked a question 21 June 2013 on OTN about finding the best fantasy football team in SQL, Processing Cost - How to catch a soccer team with the highest combined score?. I saw that this was another knapsack problem, of the single container type. I had solved that problem on the forum before, and here, A Simple SQL Solution for the Knapsack Problem (SKP-1), so I decided to adapt the solution for this case. This is in fact a more general form of the problem, wherein the items now have categories, with constraints on the numbers in each category, and on the overall number of items. The first solution I posted provided an exact solution, as in the above article, and performed well enough on the simple sample data, returning in a few seconds. However, the poster reported that the query was still running on his full data set after a couple of hours. I therefore decided to look for a mechanism to reduce the work done by the query on what is a hard combinatorial problem, and to return 'good' solutions in a practical amount of time, but without guaranteeing optimality (I recently provided solutions like this for a related problem, SQL for the Balanced Number Partitioning Problem).

This article provides the SQL that does this, and also a PL/SQL package containing a pipelined function that applies a slightly different algorithm; the latter is also practical, although it proved less efficient on my test problems.

Note that this article is a re-write of the article I published 22 June, which had the exact solution approach, and also an earlier data model, closer to the poster's own.

Test Problems

I used two test problems.

Test Problem 1: Brazilian League
The first problem was supplied by the OTN poster and appears to be based on a Brazilian league. It has 114 players, in seven positions (one being coach), with twelve players forming a team. The problem is to find the team with maximum total player points within a given maximum price, and matching the positional constraints:

Input positions

ID MIN_PLAYERS MAX_PLAYERS
-- ----------- -----------
AL          12          12
CB           2           3
CO           1           1
FW           1           3
GK           1           1
MF           3           5
WB           0           2

6 rows selected.

Input players

ID  CLUB_NAME                      PLAYER_NAME                    PO      PRICE AVG_POINTS APPEARANCES      PRF_R      VFP_R      PRC_R
--- ------------------------------ ------------------------------ -- ---------- ---------- ----------- ---------- ---------- ----------
038 Portuguesa                     Ivan                           WB        755       1320         100          1          2         20
001 Atlético-PR                    Éderson                        FW       1712       1012         500          2         22         97
002 Vitória                        Maxi Biancucchi                FW       1962       1005         400          3         33        103
003 Fluminense                     Rafael Sobis                   FW       2303        955         400          4         47        112
098 Fluminense                     Digão                          CB        931        927         300          5          5         34
058 Internacional                  Fred                           MF       3028        892         500          6         92        114
059 Grêmio                         Zé Roberto                     MF       2593        878         400          7         73        113
039 Vasco                          Elsinho                        WB       1468        850         400          8         25         83
004 Bahia                          Fernandão                      FW       1328        822         500          9         19         70
060 Internacional                  Otavinho                       MF        762        807         300         10          4         21
078 Flamengo                       Jaime De AlMFda                CO       1156        803         100         11         12         52
022 Vitória                        Wilson                         GK       1239        794         500         12         17         59
021 Cruzeiro                       Fábio                          GK       2090        794         500         12         59        106
023 Coritiba                       Vanderlei                      GK       1858        776         500         14         45        101
005 São Paulo                      Luis Fabiano                   FW       2154        758         400         15         67        107
040 Cruzeiro                       Egídio                         WB       1482        752         500         16         34         84
041 Fluminense                     Carlinhos                      WB       1240        693         300         17         26         60
099 Flamengo                       Samir                          CB        267        680         100         18          1          1
061 Vasco                          Carlos Alberto                 MF       1501        675         200         19         42         85
006 Botafogo                       Rafael Marques                 FW       1974        668         500         20         74        105
062 Cruzeiro                       Nilton                         MF       2239        646         500         21         95        110
100 Cruzeiro                       Dedé                           CB       2254        640         500         22         97        111
063 Coritiba                       Júnior Urso                    MF       1438        622         500         23         43         81
064 Crisciúma                      João Vitor                     MF       1327        604         500         24         41         69
101 São Paulo                      Lúcio                          CB       2171        602         500         25         99        108
007 Cruzeiro                       Dagoberto                      FW       2211        594         500         26        102        109
102 Grêmio                         Bressan                        CB       1085        590         400         27         28         48
103 Atlético-PR                    Manoel                         CB       1699        588         500         28         70         96
065 Corinthians                    Guilherme                      MF        883        587         400         29         14         32
104 Ponte Preta                    Cléber                         CB       1461        578         500         30         55         82
008 Náutico                        Rogério                        FW       1062        570         500         31         29         44
066 Corinthians                    Ralf                           MF       1965        570         500         31         93        104
067 Vitória                        Escudero                       MF       1638        568         500         33         68         93
068 Portuguesa                     Correa                         MF        844        560         400         34         15         26
042 Náutico                        Auremir                        WB        773        548         400         35         11         22
079 Cruzeiro                       Marcelo Oliveira               CO       1611        543         500         36         75         92
080 Fluminense                     Abel Braga                     CO       1751        536         400         37         84         98
105 Cruzeiro                       Bruno Rodrigo                  CB       1547        528         500         38         72         88
043 Cruzeiro                       Mayke                          WB        374        525         200         39          3          3
069 Portuguesa                     Souza                          MF       1262        517         400         40         49         62
070 Coritiba                       Alex                           MF       1698        508         500         41         88         95
009 Flamengo                       Hernane                        FW       1387        498         500         42         65         75
071 Grêmio                         Souza                          MF       1380        498         400         42         64         74
106 Santos                         Edu Dracena                    CB       1682        497         300         44         90         94
010 Crisciúma                      Lins                           FW       1840        490         500         45        103        100
011 Santos                         Neilton                        FW        638        488         400         46          9         11
012 Fluminense                     Samuel                         FW       1001        487         300         47         36         37
072 Ponte Preta                    Cicinho                        MF       1142        472         500         48         48         51
024 Atlético-MG                    Victor                         GK       1163        467         400         49         52         53
045 Atlético-MG                    Richarlyson                    WB       1020        467         300         49         40         38
044 Portuguesa                     Luis Ricardo                   WB        858        467         300         49         27         28
013 Ponte Preta                    Chiquinho                      FW        997        464         500         52         38         36
081 Internacional                  Dunga                          CO       1422        463         500         53         79         80
047 São Paulo                      Juan                           WB        789        457         300         54         24         23
046 Internacional                  Fabrício                       WB        876        457         400         54         31         30
014 Atlético-MG                    Luan                           FW       1318        455         400         56         71         67
048 São Paulo                      Paulo Miranda                  WB       1053        454         500         57         44         41
049 Flamengo                       João Paulo                     WB        715        453         300         58         18         19
050 São Paulo                      Rodrigo Caio                   WB       1192        452         500         59         60         56
025 Bahia                          Marcelo Lomba                  GK       1364        450         500         60         78         71
073 Botafogo                       Fellype Gabriel                MF        860        447         400         61         32         29
082 Vitória                        Caio Júnior                    CO       1140        445         500         62         56         50
015 Ponte Preta                    William                        FW       1393        444         500         63         81         76
107 Náutico                        William Alves                  CB        556        443         300         64          8          8
083 Grêmio                         Vanderlei Luxemburgo           CO       1577        442         400         65         98         89
084 São Paulo                      Ney Franco                     CO       1515        439         500         66         94         86
074 Atlético-PR                    João Paulo                     MF       1056        438         500         67         46         42
026 Botafogo                       Renan                          GK        677        437         400         68         16         13
075 Vasco                          Sandro Silva                   MF       1076        428         500         69         53         46
108 Fluminense                     Gum                            CB       1218        422         400         70         69         58
085 Náutico                        Levi Gomes                     CO        708        420         200         71         21         18
109 Flamengo                       Wallace                        CB        429        420         200         71          6          4
051 Coritiba                       Victor Ferraz                  WB       1304        420         500         71         80         65
076 Santos                         Cícero                         MF       1415        418         500         74         91         78
027 Flamengo                       Felipe                         GK       1526        414         500         75        101         87
077 Fluminense                     Wagner                         MF        855        413         300         76         37         27
052 Bahia                          Jussandro                      WB        694        410         500         77         23         16
110 Náutico                        João Filipe                    CB        547        410         400         77         10          7
016 Botafogo                       Vitinho                        FW       1020        404         500         79         54         38
053 Santos                         Rafael Galhardo                WB       1288        404         500         79         83         64
111 Grêmio                         Werley                         CB       1590        403         400         81        105         90
055 Náutico                        Maranhão                       WB        653        402         500         82         20         12
054 Goiás                          William Matheus                WB        587        402         500         82         13          9
112 Corinthians                    Gil                            CB       1323        398         500         84         85         68
113 Vitória                        Gabriel Paulista               CB       1177        394         500         85         76         54
086 Atlético-PR                    Ricardo Drubscky               CO        796        392         500         86         35         24
087 Coritiba                       Marquinhos Santos              CO       1059        389         500         87         62         43
017 Coritiba                       Deivid                         FW       1590        376         500         88        107         90
028 Grêmio                         Dida                           GK       1132        375         400         89         77         49
114 Goiás                          Ernando                        CB       1024        374         500         90         63         40
029 Corinthians                    Cássio                         GK       1251        374         500         90         89         61
018 Grêmio                         Barcos                         FW       1896        367         400         92        110        102
088 Vasco                          Paulo Autuori                  CO       1313        361         500         93        100         66
030 Vasco                          Michel Alves                   GK        899        348         500         94         57         33
019 Atlético-MG                    Jô                             FW       1393        340         200         95        106         76
056 Internacional                  Gabriel                        WB       1181        338         500         96         96         55
057 Goiás                          Vítor                          WB        877        336         500         97         58         31
089 Portuguesa                     Edson Pimenta                  CO        367        326         400         98          7          2
090 Botafogo                       Oswaldo De Oliveira            CO       1077        323         500         99         87         47
031 Crisciúma                      Bruno                          GK       1066        320         500        100         86         45
092 Santos                         Claudinei Oliveira             CO       1192        317         300        101        104         56
091 Corinthians                    Tite                           CO       1368        317         500        101        108         73
020 São Paulo                      Osvaldo                        FW       1364        312         500        103        109         71
032 Internacional                  Muriel                         GK        981        310         400        104         82         35
033 Santos                         Rafael                         GK       1782        300         500        105        112         99
093 Bahia                          Cristóvão Borges               CO        827        292         500        106         66         25
094 Crisciúma                      Vadão                          CO        704        286         500        107         50         17
095 Goiás                          Enderson Moreira               CO        680        253         500        108         61         14
034 Atlético-PR                    Weverton                       GK        616        248         500        109         51         10
035 Fluminense                     Ricardo Berna                  GK        460        242         400        110         30          6
096 Atlético-MG                    Cuca                           CO       1262        232         400        111        111         62
036 Portuguesa                     Gledson                        GK        452        210         400        112         39          5
037 São Paulo                      Rogério Ceni                   GK       1420        117         400        113        114         79
097 Ponte Preta                    Zé Sérgio                      CO        685         75         100        114        113         15

114 rows selected.

Note that I dropped the poster's formations based data model in favour of the above, more general one. I used AL as a code for team size, and chose the maximum price arbitrarily (but having an influence on results). I also multiplied the points and prices by a factor of 100 to allow me to work in integers.

Test Problem 2: English Premier League
The second problem is be based on English Premier League and I got the data from a 'scraping' web-site, https://scraperwiki.com/scrapers/fantasy_premier_league_player_stats/. There are some data quality issues with the data, but it is good enough for technical testing. I summed the players' points over the last season and took their values at the last week as prices.

After excluding zero-point players, there remained 576 players, of five positions, with eleven players forming a team, and the problem is the same, with the positional constraints:

Input positions

ID MIN_PLAYERS MAX_PLAYERS
-- ----------- -----------
AL          11          11
DF           3           5
FW           1           3
GK           1           1
MF           2           5

Input players

        ID CLUB_NAME       PLAYER_NAME          PO      PRICE AVG_POINTS APPEARANCES      PRF_R      VFP_R      PRC_R
---------- --------------- -------------------- -- ---------- ---------- ----------- ---------- ---------- ----------
       661 Tottenham       Gareth Bale          MF        111        240          38          1         36        573
       286 Liverpool       Luis Suarez          FW        105        213          38          2         62        572
        30 Arsenal         Santi Santi Cazorla  MF         97        198          36          3         57        569
       149 Chelsea         Juan Mata            MF        102        190          36          4         90        571
       265 Liverpool       Steven Gerrard       MF         92        187          38          5         60        562
       533 Southampton     Rickie Lambert       FW         69        178          37          6          5        513
       165 Everton         Leighton Baines      DF         78        173          38          7         25        537
       318 Man City        Carlos Tevez         FW         92        172          38          8         87        562
       139 Chelsea         Eden Hazard          MF         96        171          35          9         99        568
       641 Swansea         Miguel Michu         MF         79        169          36         10         41        542
       177 Everton         Marouane Fellaini    MF         73        168          38         11         14        527
        47 Aston Villa     Christian Benteke    FW         74        166          35         12         20        530
       204 Fulham          Dimitar Berbatov     FW         71        161          37         13         18        523
       720 West Brom       Romelu Lukaku        FW         66        157          37         14          8        498
       314 Man City        David Silva          MF         92        154          38         15        117        562
       298 Man City        Joe Hart             GK         69        154          38         15         22        513
       549 Stoke City      Asmir Begovic        GK         56        154          40         15          1        421
       428 Norwich         Robert Snodgrass     MF         62        152          38         18          7        474
       332 Man Utd         Patrice Evra         DF         73        152          38         18         51        527
       126 Chelsea         Demba Ba             FW         78        149          37         20         77        537
       616 Sunderland      Stephane Sessegnon   MF         67        148          38         21         26        504
       575 Stoke City      Jonathan Walters     MF         63        147          40         22         13        482
       770 West Ham        Kevin Nolan          MF         61        145          36         23          9        465
       354 Man Utd         Wayne Rooney         FW        116        141          37         24        201        575
       322 Man City        Yaya Yaya Toure      MF         82        141          37         24        108        548
       268 Liverpool       Glen Johnson         DF         65        141          37         24         35        491
       760 West Ham        Jussi Jaaskelainen   GK         52        139          36         27          2        361
       198 Everton         Steven Pienaar       MF         66        139          38         27         46        498
       609 Sunderland      Simon Mignolet       GK         53        139          38         27          3        379
       598 Sunderland      Adam Johnson         MF         68        138          37         30         61        508
       726 West Brom       James Morrison       MF         57        135          39         31         11        429
       569 Stoke City      Ryan Shawcross       DF         56        133          40         32         10        421
       248 Liverpool       Daniel Agger         DF         64        133          38         32         52        488
       270 Liverpool       Sanchez Jose Enrique DF         61        133          37         32         33        465
       239 Fulham          Mark Schwarzer       GK         51        133          38         32          4        344
       800 Wigan           Arouna Kone          FW         69        131          37         36         78        513
       684 Tottenham       Aaron Lennon         MF         71        131          38         36         91        523
       161 Chelsea         Fernando Torres      FW         93        131          36         36        165        566
       552 Stoke City      Peter Crouch         FW         60        131          40         36         31        458
       594 Sunderland      Steven Fletcher      FW         67        131          36         36         71        504
       295 Man City        Edin Dzeko           FW         68        130          38         41         76        508
       700 Tottenham       Jan Vertonghen       DF         68        129          37         42         80        508
       132 Chelsea         Petr Cech            GK         64        129          38         42         64        488
       144 Chelsea         Frank Lampard        MF         85        128          36         44        150        553
       289 Man City        Sergio Aguero        FW        111        127          39         45        217        573
       278 Liverpool       Jose Reina           GK         58        126          38         46         34        438
       628 Swansea         Jonathan De Guzman   MF         57        122          36         47         39        429
       667 Tottenham       Jermain Defoe        FW         79        122          37         47        137        542
       723 West Brom       Gareth McAuley       DF         52        122          38         47         12        361
       802 Wigan           Shaun Maloney        MF         54        121          37         50         21        395
       405 Norwich         Sebastien Bassong    DF         53        121          37         50         16        379
       186 Everton         Phil Jagielka        DF         59        120          38         52         59        449
       558 Stoke City      Robert Huth          DF         55        120          40         52         32        409
       353 Man Utd         Rafael Rafael        DF         61        119          38         54         72        465
       771 West Ham        Joey O'Brien         DF         48        119          36         54          6        269
       196 Everton         Leon Osman           MF         62        119          37         54         75        474
       650 Swansea         Wayne Routledge      MF         53        118          36         57         24        379
       323 Man City        Pablo Zabaleta       DF         64        117          38         58         92        488
       669 Tottenham       Clint Dempsey        MF         89        116          37         59        186        557
       612 Sunderland      John O'Shea          DF         51        115          38         60         19        344
       374 Newcastle       Papiss Cisse         FW         87        115          39         60        182        556
       142 Chelsea         Branislav Ivanovic   DF         69        114          36         62        120        513
       211 Fulham          Damien Duff          MF         58        114          38         62         69        438
       364 Man Utd         David de Gea         GK         58        114          38         62         69        438
       185 Everton         Tim Howard           GK         53        113          38         65         43        379
       154 Chelsea         Emboaba Oscar        MF         79        113          35         65        163        542
       602 Sunderland      Sebastian Larsson    MF         59        112          38         67         79        449
       719 West Brom       Shane Long           FW         58        110          38         68         81        438
       413 Norwich         Grant Holt           FW         59        110          38         68         89        449
       713 West Brom       Ben Foster           GK         51        109          39         70         42        344
       536 Southampton     Jason Puncheon       MF         47        107          37         71         17        238
       232 Fulham          Sascha Riether       DF         48        107          37         71         23        269
       145 Chelsea         David Luiz           DF         67        107          36         71        132        504
       784 Wigan           Jean Beausejour      MF         53        106          38         74         65        379
        60 Aston Villa     Bradley Guzan        GK         48        106          38         74         27        269
       293 Man City        Gael Clichy          DF         58        106          38         74         93        438
       476 QPR             Adel Taarabt         MF         53        105          38         77         66        379
       804 Wigan           James McCarthy       MF         48        105          38         77         30        269
       595 Sunderland      Craig Gardner        MF         49        104          38         79         44        293
       131 Chelsea         Gary Cahill          DF         60        104          38         79        106        458
       423 Norwich         Anthony Pilkington   MF         55        104          38         79         82        409
       541 Southampton     Morgan Schneiderlin  MF         48        103          37         82         38        269
       412 Norwich         Javier Garrido       DF         47        103          38         82         29        238
       414 Norwich         Wes Hoolahan         MF         55        103          38         82         86        409
       134 Chelsea         Ashley Cole A        DF         63        103          36         82        123        482
       701 Tottenham       Kyle Walker          DF         61        103          37         82        115        465
       540 Southampton     Jay Rodriguez        FW         52        103          37         82         67        361
       173 Everton         Sylvain Distin       DF         54        102          38         88         83        395
       236 Fulham          Bryan Ruiz           FW         50        102          38         88         58        313
       501 Reading         Jobi McAnuff         MF         47        101          36         90         37        238
       776 West Ham        Winston Reid         DF         48        101          36         90         47        269
       576 Stoke City      Glenn Whelan         MF         49        101          40         90         54        293
       233 Fulham          John Arne Riise      DF         52        100          38         93         74        361
       358 Man Utd         Antonio Valencia     MF         82        100          38         93        200        548
       331 Man Utd         Jonny Evans J        DF         53         99          37         95         88        379
       285 Liverpool       Daniel Sturridge     FW         74         99          35         95        178        530
       187 Everton         Nikica Jelavic       FW         77         98          37         97        190        534
       498 Reading         Adam Le Fondre       FW         44         97          36         98         28        156
       261 Liverpool       Stewart Downing      MF         57         97          37         98        109        429
       417 Norwich         Bradley Johnson      MF         47         97          38         98         53        238
       420 Norwich         Russell Martin R     DF         42         96          38        101         15         82
       648 Swansea         Angel Rangel         DF         47         96          36        101         56        238
       769 West Ham        Mark Noble           MF         46         96          36        101         50        216
       267 Liverpool       Jordan Henderson     MF         48         95          38        104         68        269
       156 Chelsea         Nascimento Ramires   MF         62         95          35        104        140        474
       635 Swansea         Pablo Hernandez      MF         59         95          33        104        130        449
       606 Sunderland      James McClean        MF         56         95          39        104        111        421
       372 Newcastle       Yohan Cabaye         MF         65         94          38        108        158        491
       327 Man Utd         Michael Carrick      MF         59         94          38        108        133        449
       631 Swansea         Nathan Dyer          MF         50         94          36        108         84        313
       305 Man City        James Milner         MF         61         93          38        111        143        465
       532 Southampton     Adam Lallana         MF         56         93          37        111        118        421
       550 Stoke City      Geoff Cameron        DF         43         92          38        113         40        114
       627 Swansea         Ben Davies           DF         44         92          35        113         49        156
       334 Man Utd         Rio Ferdinand        DF         58         92          38        113        135        438
       705 West Brom       Chris Brunt          MF         53         92          37        113        105        379
       731 West Brom       Jonas Olsson         DF         49         92          39        113         85        293
       786 Wigan           Emmerson Boyce       DF         47         91          38        118         73        238
       338 Man Utd         Javier Hernandez     FW         65         90          37        119        170        491
       789 Wigan           Franco Di Santo      FW         52         90          38        119        107        361
       191 Everton         Kevin Mirallas       FW         66         90          36        119        173        498
       658 Swansea         Ashley Williams      DF         49         89          36        122         94        293
       657 Swansea         Michel Vorm          GK         51         89          37        122        104        344
       686 Tottenham       Hugo Lloris          GK         58         89          34        122        139        438
       282 Liverpool       Martin Skrtel        DF         56         89          38        122        134        421
       761 West Ham        Matthew Jarvis       MF         55         89          35        122        129        409
       164 Everton         Victor Anichebe      FW         43         88          38        127         55        114
       735 West Brom       Liam Ridgewell       DF         48         87          38        128         97        269
       754 West Ham        Guy Demel            DF         41         87          36        128         45         60
       433 Norwich         Michael Turner       DF         41         86          38        130         48         60
       747 West Ham        Andy Carroll         FW         82         86          36        130        237        548
       547 Stoke City      Charlie Adam         MF         65         85          38        132        184        491
       291 Man City        Gareth Barry         MF         52         85          38        132        124        361
       537 Southampton     Gaston Ramirez       MF         52         85          34        132        124        361
       302 Man City        Vincent Kompany      DF         70         85          38        132        202        521
       383 Newcastle       Jonas Gutierrez      MF         55         84          39        136        142        409
       341 Man Utd         Shinji Kagawa        MF         79         84          37        136        235        542
       306 Man City        Samir Nasri          MF         81         83          37        138        243        546
       125 Chelsea         Cesar Azpilicueta    DF         56         83          34        138        154        421
       519 Southampton     Nathaniel Clyne      DF         41         83          37        138         63         60
        78 Aston Villa     Ashley Westwood      MF         49         83          35        138        112        293
       445 QPR             Soares Cesar         GK         47         83          35        138        100        238
       241 Fulham          Steve Sidwell        MF         49         83          38        138        112        293
       755 West Ham        Mohamed Diame        MF         47         83          36        138        100        238
       564 Stoke City      Steven Nzonzi        MF         50         81          35        145        128        313
       284 Liverpool       Raheem Sterling      MF         46         81          37        145        102        216
       359 Man Utd         Robin Van Persie     FW        137         80          12        147        330        576
       727 West Brom       Youssouf Mulumbu     MF         53         80          39        147        149        379
        77 Aston Villa     Andreas Weimann      FW         51         80          31        147        136        344
       782 Wigan           Ali Al-Habsi         GK         49         80          38        147        126        293
       597 Sunderland      Danny Graham         FW         54         79          38        151        156        395
       803 Wigan           James McArthur       MF         54         78          38        152        159        395
       224 Fulham          Alex Kacaniklic      MF         43         78          38        152         95        114
       591 Sunderland      Carlos Cuellar       DF         43         78          37        152         95        114
       303 Man City        Joleon Lescott       DF         58         77          38        155        179        438
       668 Tottenham       Mousa Dembele        MF         58         77          37        155        179        438
       696 Tottenham       Gylfi Sigurdsson     MF         78         76          37        157        260        537
       511 Reading         Hal Robson-Kanu      MF         42         76          36        157         98         82
       307 Man City        Matija Nastasic      DF         53         76          34        157        161        379
       386 Newcastle       Tim Krul             GK         51         75          38        160        155        344
       522 Southampton     Steven Davis         MF         45         74          37        161        122        183
       730 West Brom       Peter Odemwingie     FW         69         74          39        161        233        513
       503 Reading         Garath McCleary      MF         44         73          36        163        119        156
       221 Fulham          Brede Hangeland      DF         48         73          38        163        146        269
       589 Sunderland      Jack Colback         MF         45         73          38        163        127        183
       370 Newcastle       Hatem Ben Arfa       MF         73         72          38        166        252        527
       392 Newcastle       Davide Santon        DF         47         72          39        166        141        238
       415 Norwich         Jonathan Howson      MF         45         72          39        166        131        183
       496 Reading         Jimmy Kebe           MF         41         72          36        166        103         60
       659 Tottenham       Emmanuel Adebayor    FW         91         71          37        170        297        560
       235 Fulham          Hugo Rodallega       FW         54         71          37        170        183        395
       172 Everton         Seamus Coleman       MF         46         71          38        170        138        216
       792 Wigan           Maynor Figueroa      DF         43         71          38        170        121        114
       509 Reading         Pavel Pogrebnyak     FW         42         71          36        170        114         82
       560 Stoke City      Kenwyne Jones        FW         50         70          40        175        166        313
       194 Everton         Steven Naismith      FW         59         70          37        175        207        449
       328 Man Utd         Tom Cleverley        MF         56         70          37        175        194        421
       469 QPR             Ryan Nelsen          DF         41         69          38        178        116         60
       259 Liverpool       Phillippe Coutinho   MF         71         69          13        178        261        523
       481 QPR             Bobby Zamora         FW         61         68          39        180        226        465
       546 Southampton     Maya Yoshida         DF         45         68          34        180        148        183
       524 Southampton     Jose Fonte           DF         40         68          37        180        110         34
       425 Norwich         John Ruddy           GK         44         67          38        183        145        156
       360 Man Utd         Nemanja Vidic        DF         66         67          38        183        245        498
       781 West Ham        Ricardo Vaz Te       FW         51         66          36        185        188        344
       663 Tottenham       Steven Caulker       DF         44         66          37        185        151        156
       337 Man Utd         Ryan Giggs           MF         60         65          38        187        232        458
        13 Arsenal         Olivier Giroud       FW         77         65          18        187        281        534
       255 Liverpool       Jamie Carragher      DF         50         65          38        187        187        313
       467 QPR             Stephane Mbia        DF         49         65          35        187        181        293
       499 Reading         Mikele Leigertwood   MF         45         65          36        187        159        183
       462 QPR             Clint Hill           DF         43         64          38        192        153        114
       520 Southampton     Jack Cork            MF         44         64          37        192        157        156
       666 Tottenham       Michael Dawson       DF         45         64          38        192        164        183
       624 Swansea         Leon Britton         MF         42         64          36        192        144         82
       715 West Brom       Zoltan Gera          MF         47         64          39        192        175        238
       561 Stoke City      Michael Kightly      MF         51         64          38        192        193        344
       625 Swansea          Chico               DF         46         64          36        192        169        216
       718 West Brom       Billy Jones          DF         44         63          38        199        162        156
       387 Newcastle       Sylvain Marveaux     MF         41         62          38        200        147         60
       463 QPR             David Hoilett        MF         56         62          38        200        229        421
       556 Stoke City      Matthew Etherington  MF         59         61          40        202        241        449
       230 Fulham          Mladen Petric        FW         54         61          37        202        221        395
       478 QPR             Armand Traore        DF         48         61          38        202        191        269
       636 Swansea         Sung-Yeung Ki        MF         60         60          34        205        246        458
       544 Southampton     Luke Shaw            DF         40         60          37        205        151         34
       779 West Ham        Matthew Taylor       MF         46         60          36        205        185        216
       614 Sunderland      Danny Rose           MF         44         60          38        205        173        156
       407 Norwich         Mark Bunn            GK         43         60          34        205        167        114
       160 Chelsea         John Terry           DF         65         59          36        210        269        491
       796 Wigan           Jordi Gomez          MF         52         59          38        210        220        361
       488 Reading         Adam Federici        GK         43         59          36        210        172        114
       363 Man Utd         Ashley Young         MF         82         58          37        213        307        548
       431 Norwich         Alexander Tettey     MF         43         58          36        213        176        114
       710 West Brom       Graham Dorrans       MF         50         58          39        213        213        313
       695 Tottenham       Raniere Sandro       MF         47         58          38        213        198        238
        75 Aston Villa     Ron Vlaar            DF         45         58          38        213        189        183
       361 Man Utd         Danny Welbeck        FW         78         56          37        218        304        537
       466 QPR             Jamie Mackie         FW         50         56          38        218        223        313
       744 West Brom       Claudio Yacob        MF         49         56          37        218        218        293
       516 Southampton     Artur Boruc          GK         45         56          30        218        196        183
       297 Man City        Francisco Garcia     MF         50         56          34        218        223        313
       559 Stoke City      Cameron Jerome       FW         50         55          38        223        231        313
       787 Wigan           Gary Caldwell        DF         47         55          38        223        211        238
       301 Man City        Aleksandar Kolarov   DF         55         55          38        223        246        409
       500 Reading         Adrian Mariappa      DF         39         54          36        226        170         19
        41 Aston Villa     Gabriel Agbonlahor   FW         68         54          21        226        295        508
        38 Arsenal         Theo Walcott         MF         90         53          12        228        329        559
       474 QPR             Loic Remy            FW         54         53          16        228        253        395
       394 Newcastle       Moussa Sissoko       MF         54         53          15        228        253        395
       274 Liverpool       Leiva Lucas          MF         46         53          38        228        216        216
       506 Reading         Sean Morrison        DF         38         53          29        228        168          4
       495 Reading         Jem Karacan          MF         42         53          36        228        192         82
       153 Chelsea         Victor Moses         MF         62         53          35        228        277        474
       249 Liverpool       Joe Allen            MF         45         52          37        235        214        183
       406 Norwich         Elliott Bennett      MF         47         52          40        235        230        238
       375 Newcastle       Fabricio Coloccini   DF         49         51          38        237        240        293
       656 Swansea         Gerhard Tremmel      GK         41         51          38        237        197         60
       470 QPR             Nedum Onuoha         DF         38         51          39        237        177          4
        12 Arsenal         Kieran Gibbs         DF         53         51          15        237        262        379
        26 Arsenal         Lukas Podolski       FW         81         50          12        241        322        546
       674 Tottenham       William Gallas       DF         50         50          38        241        246        313
        23 Arsenal         Nacho Monreal        DF         52         50          13        241        263        361
       750 West Ham        Carlton Cole         FW         44         50          36        241        219        156
       222 Fulham          Aaron Hughes         DF         40         50          38        241        194         34
       772 West Ham        Gary O'Neil          MF         43         50          36        241        212        114
       382 Newcastle       Yoan Gouffran        FW         62         50          15        241        290        474
       477 QPR             Andros Townsend      MF         44         49          38        248        227        156
       369 Newcastle       Vurnon Anita         MF         44         49          38        248        227        156
       446 QPR             Djibril Cisse        FW         58         49          39        248        280        438
       182 Everton         Johnny Heitinga      DF         50         49          38        248        255        313
       225 Fulham          Giorgos Karagounis   MF         47         49          34        248        239        238
       356 Man Utd         Chris Smalling       DF         45         48          38        253        234        183
       570 Stoke City      Ryan Shotton         MF         46         48          38        253        238        216
       319 Man City        Kolo Toure           DF         51         48          39        253        267        344
       492 Reading         Danny Guthrie        MF         41         48          36        253        210         60
       578 Stoke City      Andy Wilkinson       DF         40         48          40        253        204         34
       502 Reading         Alex McCarthy        GK         40         48          36        253        204         34
       690 Tottenham       Kyle Naughton        DF         39         48          37        253        199         19
       810 Wigan           Ivan Ramis           DF         42         47          37        260        225         82
       457 QPR             Esteban Granero      MF         52         47          36        260        270        361
       180 Everton         Darron Gibson        MF         47         47          38        260        246        238
       393 Newcastle       Danny Simpson        DF         46         46          38        263        246        216
       440 QPR             Jose Bosingwa        DF         48         46          38        263        264        269
       780 West Ham        James Tomkins        DF         41         46          36        263        222         60
       579 Stoke City      Marc Wilson          DF         39         46          38        263        209         19
       491 Reading         Chris Gunter         DF         38         46          36        263        203          4
       240 Fulham          Philippe Senderos    DF         47         45          38        268        265        238
       202 Fulham          Chris Baird          DF         39         45          38        268        215         19
       288 Liverpool       Andre Wisdom         DF         38         45          32        268        208          4
       207 Fulham          Ashkan Dejagah       MF         55         45          34        268        288        409
       152 Chelsea          Mikel               MF         43         44          36        272        244        114
       398 Newcastle       Steven Taylor S      DF         46         44          38        272        266        216
       493 Reading         Ian Harte            DF         37         44          36        272        206          1
       471 QPR             Ji-Sung Park         MF         52         44          38        272        279        361
       479 QPR             Shaun Wright-Phillip MF         48         44          39        272        268        269
                           s

       691 Tottenham       Scott Parker         MF         52         43          37        277        285        361
        11 Arsenal         Yao Gervinho         MF         68         42          12        278        321        508
       577 Stoke City      Dean Whitehead       MF         42         42          40        278        246         82
       453 QPR             Fabio Fabio          DF         40         42          38        278        236         34
         2 Arsenal         Mikel Arteta         MF         75         41          13        281        339        533
       325 Man Utd         Oliveira Anderson    MF         51         41          38        281        292        344
       712 West Brom       Marc-Antoine Fortune FW         48         41          38        281        278        269
       752 West Ham        James Collins        DF         46         41          14        281        274        216
       195 Everton         Phil Neville         MF         41         40          38        285        256         60
       660 Tottenham       Benoit Assou-Ekotto  DF         60         40          38        285        313        458
        21 Arsenal         Per Mertesacker      DF         53         39          13        287        303        379
       513 Reading         Nicky Shorey         DF         38         39          36        287        242          4
        16 Arsenal         Carl Jenkinson       DF         40         39          13        287        257         34
       531 Southampton     Jos Hooiveld         DF         40         39          37        287        257         34
       436 Norwich         Steven Whittaker     DF         40         39          38        287        257         34
       643 Swansea         Luke Moore           FW         43         38          36        292        275        114
       399 Newcastle       Cheick Tiote         MF         48         38          38        292        296        269
       326 Man Utd         Alexander Buttner    DF         50         37          36        294        302        313
       458 QPR             Rob Green            GK         41         37          38        294        271         60
       494 Reading         Noel Hunt            FW         46         37          36        294        291        216
       678 Tottenham       Tom Huddlestone      MF         45         37          37        294        287        183
       389 Newcastle       James Perch          DF         44         37          38        294        283        156
       655 Swansea         Dwight Tiendalli     DF         45         36          32        299        293        183
        69 Aston Villa     Matthew Lowton       DF         45         36          12        299        293        183
        37 Arsenal         Thomas Vermaelen     DF         67         36          12        299        343        504
       128 Chelsea         Ryan Bertrand        DF         39         35          36        302        272         19
       751 West Ham        Joe Cole             MF         51         35          37        302        310        344
       805 Wigan           Callum McManaman     FW         45         35          38        302        298        183
       401 Newcastle       Mike Williamson      DF         40         35          38        302        276         34
       508 Reading         Alex Pearce          DF         38         34          36        306        273          4
       350 Man Utd         Luis Nani            MF         82         34          38        306        369        548
       368 Newcastle       Shola Ameobi         FW         51         34          38        306        313        344
       610 Sunderland      Alfred N'Diaye       MF         42         34          17        306        289         82
       584 Sunderland      Titus Bramble        DF         40         33          38        310        286         34
       311 Man City        Micah Richards       DF         57         33          38        310        332        429
       618 Sunderland      David Vaughan        MF         49         33          38        310        312        293
       766 West Ham        George McCartney     DF         38         32          36        313        282          4
       528 Southampton     Guly Guilherme       MF         47         32          37        313        311        238
       753 West Ham        Jack Collison        MF         46         32          36        313        309        216
       464 QPR             Jermaine Jenas       MF         42         32          38        313        300         82
       489 Reading         Kaspars Gorkss       DF         37         31          36        317        284          1
       521 Southampton     Kelvin Davis         GK         41         31          37        317        301         60
        19 Arsenal         Vito Mannone         GK         40         31          36        317        299         34
       448 QPR             Shaun Derry          MF         42         30          38        320        306         82
       764 West Ham        Modibo Maiga         FW         50         30          36        320        324        313
       818 Wigan           Ben Watson           MF         50         30          38        320        324        313
       402 Newcastle       Mapou Yanga-Mbiwa    DF         49         29          15        323        328        293
       737 West Brom       Markus Rosenberg     FW         59         29          37        323        357        449
       355 Man Utd         Paul Scholes         MF         50         29          38        323        331        313
       426 Norwich         Ryan Ryan Bennett    DF         39         28          38        326        304         19
       526 Southampton     Daniel Fox           DF         40         28          37        326        308         34
        51 Aston Villa     Ciaran Clark         DF         44         28          12        326        318        156
       290 Man City        Mario Balotelli      FW         86         28          38        326        389        554
       454 QPR             Alejandro Faurlin    MF         47         28          38        326        326        238
       304 Man City        Sisenando Maicon     DF         62         28          34        326        363        474
        45 Aston Villa     Joe Bennett          DF         44         28          36        326        318        156
       791 Wigan           Roger Espinoza       MF         41         27          16        333        315         60
        44 Aston Villa     Barry Bannan         MF         47         27          12        333        333        238
       651 Swansea         Itay Shechter        FW         50         27          35        333        342        313
        46 Aston Villa     Darren Bent          FW         78         27          18        333        384        537
       280 Liverpool       Nuri Sahin           MF         54         27          35        333        352        395
        27 Arsenal         Aaron Ramsey         MF         54         27          12        333        352        395
       231 Fulham          Kieran Richardson    MF         53         27          36        333        351        379
       527 Southampton     Paulo Gazzaniga      GK         40         26          37        340        316         34
       497 Reading         Stephen Kelly        DF         40         26          38        340        316         34
       783 Wigan           Antolin Alcaraz      DF         42         26          38        340        320         82
       340 Man Utd         Phil Jones           DF         57         26          38        340        362        429
       421 Norwich         Steve Morison        FW         49         26          38        340        345        293
       346 Man Utd         Anders Lindegaard    GK         51         26          38        340        350        344
       312 Man City        Jack Rodwell         MF         46         26          37        340        336        216
       388 Newcastle       Gabriel Obertan      MF         41         25          39        347        323         60
       728 West Brom       Boaz Myhill          GK         44         25          39        347        334        156
       582 Sunderland      Phil Bardsley        DF         44         25          37        347        334        156
        56 Aston Villa     Karim El Ahmadi      MF         42         25          16        347        327         82
       798 Wigan           David Jones          MF         43         24          39        351        337        114
       514 Reading         Jay Tabb             MF         43         24          36        351        337        114
       269 Liverpool       Brad Jones           GK         44         24          37        351        340        156
        62 Aston Villa     Brett Holman         MF         55         24          12        351        364        409
       482 Reading         Hope Akpan           MF         45         24          17        351        344        183
       376 Newcastle       Mathieu Debuchy      DF         47         24          17        351        348        238
       416 Norwich         Simeon Jackson       FW         47         24          38        351        348        238
       449 QPR             Samba Diakite        MF         44         24          39        351        340        156
       672 Tottenham       Brad Friedel         GK         48         23          37        359        360        269
       642 Swansea         Garry Monk           DF         42         22          36        360        346         82
         5 Arsenal         Alex Chamberlain     MF         69         22          13        360        391        513
       621 Swansea         Kemy Agustien        MF         45         22          36        360        358        183
       545 Southampton     James Ward-Prowse    MF         43         22          37        360        347        114
       371 Newcastle       Gael Bigirimana      MF         43         21          38        364        359        114
       281 Liverpool       Jonjo Shelvey        MF         51         21          39        364        371        344
       212 Fulham          Urby Emanuelson      MF         46         21          13        364        361        216
       377 Newcastle       Rob Elliot           GK         40         20          38        367        352         34
       734 West Brom       Steven Reid          MF         47         20          39        367        367        238
       812 Wigan           Joel Robles          GK         40         20          15        367        352         34
       815 Wigan           Ronnie Stam          DF         38         19          38        370        352          4
        17 Arsenal         Laurent Koscielny    DF         53         19          12        370        382        379
        64 Aston Villa     Stephen Ireland      MF         50         19          12        370        377        313
       733 West Brom       Goran Popov          DF         44         19          34        370        366        156
       510 Reading         Jason Roberts        FW         45         18          36        374        372        183
       263 Liverpool        Fernandez Saez      FW         47         18          33        374        375        238
       254 Liverpool       Fabio Borini         FW         72         18          37        374        409        526
       315 Man City        Scott Sinclair       MF         60         18          37        374        392        458
       197 Everton         Bryan Oviedo         MF         48         18          34        374        379        269
       455 QPR             Anton Ferdinand      DF         41         17          39        379        369         60
        67 Aston Villa     Eric Lichaj          DF         43         17          12        379        373        114
       588 Sunderland      Lee Cattermole       MF         43         17          38        379        373        114
       654 Swansea         Neil Taylor          DF         45         17          36        379        378        183
       677 Tottenham       Lewis Holtby         MF         63         17          14        379        401        482
       586 Sunderland      Fraizer Campbell     FW         49         17          37        379        383        293
       742 West Brom       Jerome Thomas        MF         51         17          39        379        386        344
         7 Arsenal         Vassiriki Diaby      MF         61         17          12        379        398        465
       127 Chelsea         Yossi Benayoun       MF         61         17          36        379        398        465
        29 Arsenal         Bacary Sagna         DF         47         17          15        379        381        238
       404 Norwich         Leon Barnett         DF         37         16          38        389        365          1
       210 Fulham          Mahamadou Diarra     MF         47         16          37        389        385        238
        74 Aston Villa     Yacouba Sylla        MF         42         16          14        389        376         82
       599 Sunderland      Matthew Kilgallon    DF         38         16          37        389        368          4
       685 Tottenham       Jake Livermore       MF         41         15          39        393        380         60
       209 Fulham          Clint Dempsey        MF         92         15           1        393        436        562
       385 Newcastle       Steve Harper         GK         45         15          38        393        386        183
       228 Fulham          Stanislav Manolev    DF         42         14          13        396        386         82
       378 Newcastle       Shane Ferguson       MF         43         14          38        396        389        114
       183 Everton         Tony Hibbert         DF         50         14          38        396        396        313
       535 Southampton     Emmanuel Mayuka      FW         48         14          35        396        394        269
       441 QPR             Jay Bothroyd         FW         47         14          40        396        393        238
        48 Aston Villa     Jordan Bowery        FW         45         13          36        401        395        183
       330 Man Utd         Jonathan Evans J     DF         48         13           1        401        400        269
       620 Sunderland      Connor Wickham       FW         50         13          37        401        405        313
       213 Fulham          Eyong Enoh           MF         50         13          13        401        405        313
        31 Arsenal         Clarindo Santos      DF         49         13          14        401        403        293
       192 Everton         Jan Mucha            GK         43         12          38        406        397        114
       217 Fulham          Emmanuel Frimpong    MF         45         12          15        406        402        183
       103 Bolton          Mark Davies M        MF         48         12           2        406        409        269
       148 Chelsea         Marko Marin          MF         66         12          35        406        429        498
       184 Everton         Thomas Hitzlsperger  MF         50         12          30        406        412        313
       335 Man Utd         Darren Fletcher      MF         54         12          38        406        416        395
       565 Stoke City      Michael Owen         FW         50         12          35        406        412        313
       740 West Brom       Gabriel Tamas        DF         42         11          39        413        404         82
       391 Newcastle       Sammy Sammy Ameobi   FW         43         11          39        413        408        114
        72 Aston Villa     Charles N'Zogbia     MF         66         11          13        413        434        498
       114 Bolton          Martin Petrov        MF         52         11           2        413        419        361
        88 Blackburn       David Hoilett        FW         55         11           3        413        423        409
       475 QPR             Tommy Smith          FW         45         11          40        413        411        183
        43 Aston Villa     Nathan Baker         DF         39         10          12        419        407         19
       662 Tottenham       Tom Carroll          MF         42         10          37        419        414         82
        39 Arsenal         Jack Wilshere        MF         63         10          12        419        441        482
        35 Arsenal         Wojciech Szczesny    GK         53         10          12        419        427        379
        85 Blackburn       Morten Gamst Gamst P MF         62         10           2        419        439        474
                           edersen

       615 Sunderland      Louis Saha           FW         49         10          37        419        422        293
       205 Fulham          Matthew Briggs       DF         39          9          38        425        415         19
       158 Chelsea         Oriol Romeu          MF         41          9          37        425        417         60
       574 Stoke City      Matthew Upson        DF         41          9          39        425        417         60
       216 Fulham          Kerim Frei           MF         43          9          37        425        420        114
       613 Sunderland      Kieran Richardson    MF         58          9           1        425        444        438
       585 Sunderland      Wes Brown            DF         46          9          38        425        424        216
       201 Everton         Apostolos Vellios    FW         47          9          38        425        425        238
       223 Fulham          Andrew Johnson A     FW         47          9           1        425        425        238
       257 Liverpool       Sebastian Coates     DF         44          9          39        425        421        156
       450 QPR             Kieron Dyer          MF         44          8          39        434        429        156
       785 Wigan           Mauro Boselli        FW         50          8          37        434        440        313
        54 Aston Villa     Fabian Delph         MF         46          8          12        434        432        216
       486 Reading         Shaun Cummings       DF         38          7          36        437        428          4
       273 Liverpool       Dirk Kuyt            MF         94          7           1        437        478        567
       166 Everton         Ross Barkley         MF         41          7          39        437        433         60
       797 Wigan           Angelo Henriquez     FW         42          7          16        437        434         82
       743 West Brom       George Thorne        MF         43          7          39        437        437        114
       430 Norwich         Andrew Surman        MF         43          7          38        437        437        114
       580 Stoke City      Jonathan Woodgate    DF         45          7           2        437        442        183
       352 Man Utd         Nick Powell          MF         45          7          37        437        442        183
       465 QPR             Andrew Johnson       FW         46          7          38        437        446        216
         6 Arsenal         Francis Coquelin     MF         47          7          13        437        448        238
        82 Blackburn       Scott Dann           DF         47          7           1        437        448        238
       272 Liverpool       Martin Kelly         DF         51          7          38        437        452        344
        42 Aston Villa     Marc Albrighton      MF         52          7          12        437        453        361
       109 Bolton          Ivan Klasnic         FW         59          7           2        437        456        449
       515 Reading         Stuart Taylor        GK         40          7          35        437        431         34
       162 Chelsea         Ross Turnbull        GK         39          6          36        452        445         19
       409 Norwich         Lee Camp             GK         40          6          15        452        447         34
       310 Man City        Karim Rekik          DF         43          6          38        452        450        114
         1 Arsenal         Andrey Arshavin      MF         65          6          12        452        470        491
       637 Swansea         Roland Lamah         MF         50          6          14        452        455        313
       229 Fulham          Danny Murphy         MF         61          6           1        452        464        465
         9 Arsenal         Lukasz Fabianski     GK         43          6          12        452        450        114
       539 Southampton     Frazer Richardson    DF         41          5          37        459        454         60
       680 Tottenham       Harry Kane           FW         43          5          38        459        457        114
       151 Chelsea         Raul Meireles        MF         63          5          36        459        474        482
        84 Blackburn       Mauro Formica        MF         49          5           2        459        462        293
       244 Fulham          David Stockdale      GK         43          5          38        459        457        114
       709 West Brom       Craig Dawson         DF         38          4          39        464        459          4
       439 QPR             Tal Ben Haim         DF         39          4          17        464        460         19
       801 Wigan           Adrian Lopez         DF         39          4          38        464        460         19
       777 West Ham        Jordan Spence        DF         40          4          29        464        463         34
       566 Stoke City      Wilson Palacios      MF         41          4          39        464        465         60
        52 Aston Villa     Simon Dawkins        MF         42          4          14        464        466         82
       699 Tottenham       Rafael Van der Vaart MF         89          4          37        464        509        557
       717 West Brom       Gonzalo Jara         DF         43          4          39        464        468        114
        61 Aston Villa     Chris Herd           MF         43          4          12        464        468        114
       294 Man City        Nigel De Jong        MF         44          4          38        464        471        156
        58 Aston Villa     Shay Given           GK         45          4          12        464        472        183
       107 Bolton          Jussi Jaaskelainen   GK         48          4           2        464        473        269
       251 Liverpool       Oussama Assaidi      MF         57          4          36        464        483        429
       110 Bolton          Zat Knight           DF         42          4           2        464        466         82
       538 Southampton     Ben Reeves           DF         38          3          37        478        475          4
       567 Stoke City      Jermaine Pennant     MF         50          3          38        478        491        313
       122 Chelsea         Nathan Ake           DF         40          3          20        478        477         34
       384 Newcastle       Massadio Haidara     DF         41          3          15        478        479         60
       608 Sunderland      David Meyler         MF         42          3          38        478        480         82
       140 Chelsea         Henrique Hilario     GK         42          3          38        478        480         82
       518 Southampton     Richard Chaplow      MF         42          3          37        478        480         82
       309 Man City        Abdul Razak          MF         43          3          39        478        484        114
       806 Wigan           Ryo Miyaichi         MF         43          3          12        478        484        114
       774 West Ham        Emanuel Pogatetz     DF         43          3          14        478        484        114
       373 Newcastle       Adam Campbell        FW         45          3          20        478        487        183
       756 West Ham        Alou Diarra          MF         45          3          36        478        487        183
       381 Newcastle       Dan Gosling          MF         46          3          38        478        489        216
       181 Everton         Magaye Gueye         FW         46          3          38        478        489        216
       568 Stoke City      Danny Pugh           MF         50          3           2        478        491        313
       607 Sunderland      James McFadden       FW         50          3          28        478        491        313
       487 Reading         Daniel Daniel Carric DF         38          3          17        478        475          4
                           o

       653 Swansea         Alan Tate            DF         38          2          36        495        494          4
       321 Man City        Gnegneri Yaya Toure  MF         77          2           1        495        530        534
       459 QPR             Michael Harriman     DF         39          2          38        495        496         19
       795 Wigan           Roman Golobart       DF         40          2          24        495        497         34
       605 Sunderland      Kader Mangane        DF         40          2          16        495        497         34
       400 Newcastle       Haris Vuckic         MF         42          2          40        495        499         82
       119 Bolton          Gretar Rafn Steinsso DF         42          2           2        495        499         82
                           n

       155 Chelsea         Lucas Piazon         MF         42          2          35        495        499         82
       226 Fulham          Pajtim Kasami        MF         42          2          38        495        499         82
       523 Southampton     Steve De Ridder      MF         42          2          37        495        499         82
       411 Norwich         David Fox            MF         43          2          38        495        504        114
       118 Bolton          Paul Robinson        DF         43          2           2        495        504        114
       116 Bolton          Nigel Reo-Coker      MF         44          2           1        495        506        156
       833 Wolves          Karl Henry           MF         44          2           1        495        506        156
       113 Bolton          Fabrice Muamba       MF         44          2           1        495        506        156
       707 West Brom       Simon Cox            FW         45          2          38        495        510        183
       390 Newcastle       Nile Ranger          MF         45          2          26        495        510        183
       832 Wolves          Wayne Hennessey      GK         46          2           1        495        512        216
       593 Sunderland      Ahmed Elmohamady     MF         49          2          38        495        513        293
       543 Southampton     Billy Sharp          FW         49          2          37        495        513        293
       634 Swansea         Danny Graham         FW         50          2           1        495        515        313
       679 Tottenham       Younes Kaboul        DF         50          2          38        495        515        313
        91 Blackburn       Marcus Marcus Olsson MF         50          2           1        495        515        313
       738 West Brom       Paul Scharner        MF         51          2           2        495        518        344
       115 Bolton          Darren Pratley       MF         52          2           2        495        519        361
       208 Fulham          Moussa Dembele       FW         52          2           1        495        519        361
        95 Blackburn       Jason Roberts        FW         53          2           3        495        521        379
       283 Liverpool       Jay Spearing         MF         54          2          38        495        522        395
       175 Everton         Royston Drenthe      MF         54          2           1        495        522        395
       102 Bolton          Kevin Davies K       FW         57          2           2        495        524        429
       835 Wolves          Matthew Jarvis       MF         57          2           1        495        524        429
       682 Tottenham       Niko Kranjcar        MF         60          2           1        495        526        458
        83 Blackburn       David Dunn           MF         62          2           3        495        527        474
       157 Chelsea          Ramires             MF         69          2           1        495        528        513
       694 Tottenham       Louis Saha           FW         69          2           1        495        528        513
       775 West Ham        Daniel Potts         DF         38          2          36        495        494          4
       396 Newcastle       James Tavernier      DF         39          1          38        531        531         19
       136 Chelsea         Didier Drogba        FW        101          1           1        531        576        570
        73 Aston Villa     Enda Stevens         DF         39          1          12        531        531         19
       138 Chelsea         Paulo Ferreira       DF         40          1          36        531        534         34
       242 Fulham          Alex Smith           DF         40          1          34        531        534         34
       227 Fulham          Stephen Kelly        DF         40          1           1        531        534         34
       333 Man Utd         Fabio Fabio          DF         42          1           1        531        537         82
       793 Wigan           Fraser Fyvie         MF         42          1          37        531        537         82
       176 Everton         Shane Duffy          DF         42          1          38        531        537         82
        57 Aston Villa     Gary Gardner         MF         42          1          13        531        537         82
        14 Arsenal         Serge Gnabry         MF         43          1          29        531        541        114
       443 QPR             DJ Campbell          FW         43          1          40        531        541        114
       555 Stoke City      Maurice Edu          MF         43          1          36        531        541        114
       848 Wolves          Stephen Ward         DF         43          1           1        531        541        114
       820 Wolves          Christophe Berra     DF         43          1           1        531        541        114
       846 Wolves          Richard Stearman     DF         43          1           1        531        541        114
       758 West Ham        Robert Hall          FW         43          1          34        531        541        114
       512 Reading         Dominic Samuel       FW         44          1          23        531        548        156
       419 Norwich         Chris Martin C       FW         44          1          38        531        548        156
       814 Wigan           Conor Sammon         FW         45          1          38        531        550        183
       633 Swansea         Mark Gower           MF         45          1          36        531        550        183
       553 Stoke City      Rory Delap           MF         45          1          40        531        550        183
       837 Wolves          Eggert Jonsson       MF         45          1           1        531        550        183
       206 Fulham          Simon Davies         MF         46          1          38        531        554        216
       670 Tottenham       Yago Falque          MF         46          1          37        531        554        216
        96 Blackburn       Ruben Rochina        FW         47          1           3        531        556        238
       790 Wigan           Mohamed Diame        MF         48          1           1        531        557        269
       397 Newcastle       Ryan Taylor R        DF         48          1          38        531        557        269
       583 Sunderland      Phillip Bardsley     DF         48          1           1        531        557        269
       741 West Brom       Somen Tchoyi         MF         48          1           2        531        557        269
       825 Wolves          David Edwards        MF         49          1           1        531        561        293
        53 Aston Villa     Nathan Delfouneso    FW         50          1          21        531        562        313
       841 Wolves          Nenad Milijas        MF         52          1           1        531        563        361
       827 Wolves          Steven Fletcher      FW         52          1           1        531        563        361
       839 Wolves          Michael Kightly      MF         54          1           1        531        565        395
       130 Chelsea         Jose Bosingwa        DF         55          1           1        531        566        409
        86 Blackburn       David Goodwillie     FW         55          1           2        531        566        409
       342 Man Utd         Will Keane           FW         55          1           2        531        566        409
       823 Wolves          Kevin Doyle          FW         57          1           1        531        569        429
       137 Chelsea         Michael Essien       MF         63          1          36        531        570        482
       171 Everton         Tim Cahill           MF         65          1          38        531        571        491
       692 Tottenham       Roman Pavlyuchenko   FW         70          1           1        531        572        521
       146 Chelsea         Romelu Lukaku        FW         74          1           3        531        573        530
       247 Liverpool       Charlie Adam         MF         86          1           1        531        574        554
       256 Liverpool       Andy Carroll         FW         91          1           1        531        575        560
        98 Bolton          Marcos Alonso        DF         39          1           1        531        531         19

576 rows selected.

SQL Solution with Recursive Subquery Factoring

SQL

Note that currently I have retained the fantasy league table and column names, but they could as well be the generic items and categories in place of players and teams: This is a generic solution.

VAR KEEP_NUM NUMBER
VAR MAX_PRICE NUMBER
BEGIN
  :KEEP_NUM := 40;
  :MAX_PRICE := 900;
END;
/
PROMPT Top ten solutions
WITH  /* XS_EPL */ position_counts AS (
SELECT Max (CASE id WHEN 'AL' THEN min_players END) team_size
  FROM positions
), pos_runs AS (
SELECT id, Sum (CASE WHEN id != 'AL' THEN min_players END) OVER (ORDER BY id DESC) num_remain, min_players, max_players
  FROM positions
), players_ranked AS (
SELECT id,
       position_id,
       price,
       avg_points,
       appearances,
       Row_Number() OVER (ORDER BY position_id, avg_points DESC) rnk,
       Min (price) OVER () min_price
  FROM players
), rsf (path_rnk, nxt_id, lev, tot_price, tot_profit, pos_id, n_pos, team_size, min_players, pos_path, path) AS (
SELECT 0, 0, 0, 0, 0, 'AL', 0, c.team_size, 0, CAST (NULL AS VARCHAR2(400)) pos_path, CAST (NULL AS VARCHAR2(400)) path
  FROM position_counts c
 UNION ALL
SELECT Row_Number() OVER (PARTITION BY r.pos_path || p.position_id ORDER BY r.tot_profit + p.avg_points DESC),
       p.rnk,
       r.lev + 1,
       r.tot_price + p.price,
       r.tot_profit + p.avg_points,
       p.position_id,
       CASE p.position_id WHEN r.pos_id THEN r.n_pos + 1 ELSE 1 END,
       r.team_size,
       m1.min_players,
       r.pos_path || p.position_id,
       r.path || LPad (p.id, 3, '0')
  FROM rsf r
  JOIN players_ranked p
    ON p.rnk > r.nxt_id
  JOIN pos_runs m1
    ON m1.id = p.position_id
   AND CASE p.position_id WHEN r.pos_id THEN r.n_pos + 1 ELSE 1 END <= m1.max_players
   AND r.team_size - r.lev - 1 >= m1.num_remain - CASE p.position_id WHEN r.pos_id THEN r.n_pos + 1 ELSE 1 END
   AND (r.lev = 0 OR p.position_id = r.pos_id OR r.n_pos >= r.min_players)
 WHERE r.tot_price + p.price + (r.team_size - r.lev - 1) * p.min_price <= :MAX_PRICE
   AND r.path_rnk < :KEEP_NUM
   AND r.lev < r.team_size
), paths_ranked AS (
SELECT tot_price,
       tot_profit,
       team_size,
       Row_Number () OVER (ORDER BY tot_profit DESC, tot_price) r_profit,
       path
  FROM rsf
 WHERE lev = team_size
), top_ten_paths AS (
SELECT tot_price,
       tot_profit,
       r_profit,
       path,
       player_index
  FROM paths_ranked
  CROSS JOIN (SELECT LEVEL player_index FROM position_counts CONNECT BY LEVEL <= team_size)
 WHERE r_profit <= 10
), top_ten_teams AS (
SELECT tot_price,
       tot_profit,
       r_profit,
       path,
       player_index,
       Substr (path, (player_index - 1) * 3 + 1, 3) player_id
  FROM top_ten_paths
)
SELECT  /*+ GATHER_PLAN_STATISTICS */  t.tot_profit,
       t.tot_price,
       t.r_profit rnk,
       p.position_id,
       t.player_id p_id,
       p.player_name,
       p.club_name,
       p.price,
       p.avg_points
  FROM top_ten_teams t
  JOIN players p
    ON p.id = t.player_id
ORDER BY t.tot_profit DESC, t.tot_price, t.path, p.position_id, t.player_index

How It Works

The solution approach is based on the method used to provide exact solutions for knapsack problems in my earlier article, but with a number of extensions to cater for the new category constraints, and to reduce searching to manageable proportions.

  • position_counts subquery: Gets the team size
  • pos_runs subquery: Computes the running sums of the item category minima going backwards by category id
  • players_ranked subquery: Computes a unique rank for the items, ordered by category, then profit descending
  • rsf subquery: A recursive subquery that returns a set of item sets in the form of strings of the concatenated item ids
  • rsf anchor branch: Initialises the recursion with a single record
  • rsf recursive branch: Items are joined having strictly higher rank, and such that the constraints are not violated, both at the current position and with any possible extrapolations
  • Row_Number is used to rank the records by overall profit, and the where clause excludes records from the previous iteration that have rank below an input figure;
    this exclusion is what makes the computation practical; the ranking is partitioned by the category path, which is important to avoid closing off solution paths too early
  • Item category minima are treated differently from the maxima; once a category is in a position, the subsequent positions are required to be of the same category until the minimum number is reached
  • paths_ranked subquery: Excludes records that are not of full length,, and ranks those that are by profit
  • top_ten_paths subquery: Selects the top ten paths and cross-joins them with a row-generator to provide an indexed set of records with set size cardinality for each path
  • top_ten_teams subquery: Builds the item records for each of the best sets by extracting the item id from the paths according to index
  • Main query: Joins items table to provide additional attributes

PL/SQL Recursive Solution

This is a version in the form of a pipelined function.

SQL

SELECT  /*+ GATHER_PLAN_STATISTICS XP_EPL */
       t.sol_profit, 
       t.sol_price,
       Dense_Rank() OVER (ORDER BY t.sol_profit DESC, t.sol_price) RNK,
       p.position_id,
       t.item_id, 
       p.club_name,
       p.player_name,
       p.price,
       p.avg_points
  FROM TABLE (Item_Cats.Best_N_Sets (
  p_keep_size => 10, 
  p_max_calls => 100000,
  p_n_size => 10, 
  p_max_price => 900,
  p_cat_cur => CURSOR (
  SELECT id, min_players, max_players
    FROM positions
   ORDER BY CASE WHEN id != 'AL' THEN 0 END, id
  ), 
  p_item_cur => CURSOR (
  SELECT id, price, avg_points, position_id
    FROM players
   ORDER BY position_id, avg_points DESC
  )
  )
  ) t
  JOIN players p
    ON p.id = t.item_id
  ORDER BY t.sol_profit DESC, t.sol_price, p.position_id, t.item_id

Package

CREATE OR REPLACE PACKAGE Item_Cats AS
/**************************************************************************************************

Author:         Brendan Furey
Date:           7 July 2013
Description:    Brendan's pipelined function solution for the knapsack problem with one container,
                and items having categories with validity bands, as described at
                http://aprogrammerwrites.eu/?p=878 (SQL for the Fantasy Football Knapsack Problem)

***************************************************************************************************/
TYPE sol_detail_rec_type IS RECORD (
        set_id                  NUMBER,
        item_id                 VARCHAR2(100),
        sol_price               NUMBER,
        sol_profit              NUMBER
        );
TYPE sol_detail_list_type IS VARRAY(100) OF sol_detail_rec_type;

FUNCTION Best_N_Sets (  p_keep_size     PLS_INTEGER, 
                        p_max_calls     PLS_INTEGER, 
                        p_n_size        PLS_INTEGER, 
                        p_max_price     PLS_INTEGER, 
                        p_cat_cur       SYS_REFCURSOR, 
                        p_item_cur      SYS_REFCURSOR) RETURN sol_detail_list_type PIPELINED;

END Item_Cats;
/
SHO ERR

CREATE OR REPLACE PACKAGE BODY Item_Cats AS

c_cat_all           CONSTANT VARCHAR2(3) := 'AL';
c_hash_renew_point  CONSTANT PLS_INTEGER := 1000;
--
-- Bulk collect array types
--
TYPE cat_rec_type IS RECORD (
        id                      VARCHAR2(3),
        min_items               PLS_INTEGER,
        max_items               PLS_INTEGER
        );
TYPE cat_list_type IS VARRAY(100) OF cat_rec_type;

TYPE item_cat_rec_type IS RECORD (
        id                      VARCHAR2(10),
        price                   PLS_INTEGER,
        profit                  PLS_INTEGER,
        cat_id                  VARCHAR2(3)
        );
TYPE item_cat_list_type IS VARRAY(1000) OF item_cat_rec_type;

TYPE chr_hash_type IS TABLE OF PLS_INTEGER INDEX BY VARCHAR2(30);
--
-- Input data LOL types
--
TYPE num_range_rec_type IS RECORD (
        item_beg                PLS_INTEGER,
        item_end                PLS_INTEGER
        );
TYPE num_range_list_type IS VARRAY(1000) OF num_range_rec_type;
TYPE num_list_type IS VARRAY(100) OF PLS_INTEGER;
--
-- Solution types
--
TYPE id_list_type IS VARRAY(100) OF VARCHAR2(10);
TYPE sol_rec_type IS RECORD (                       -- trial solution and record in retained array
        item_list               id_list_type,
        price                   PLS_INTEGER,
        profit                  PLS_INTEGER
        );
TYPE sol_list_type IS VARRAY(100) OF sol_rec_type;  -- retained solutions

TYPE int_hash_type IS TABLE OF PLS_INTEGER INDEX BY PLS_INTEGER;

g_keep_size                 PLS_INTEGER;
g_max_calls                 PLS_INTEGER;
g_n_size                    PLS_INTEGER;
g_max_price                 PLS_INTEGER;

g_cat_hash                  chr_hash_type;
g_item_range_list           num_range_list_type := num_range_list_type();
g_hash_buffer               int_hash_type;
g_profit_hash               int_hash_type;
g_trial_sol                 sol_rec_type;
g_sol_list                  sol_list_type := sol_list_type();
g_cat_list                  cat_list_type;
g_item_cat_list             item_cat_list_type;

g_n_cats                    PLS_INTEGER;
g_n_items                   PLS_INTEGER;
g_set_size                  PLS_INTEGER;
g_nth_profit                PLS_INTEGER := 0;
g_min_item_price            PLS_INTEGER := 1000000;
g_max_item_profit           PLS_INTEGER := 0;
g_min_price_togo            num_list_type := num_list_type();
g_max_profit_togo           num_list_type := num_list_type();
g_n_recursive_calls         PLS_INTEGER := 0;
g_n_sols                    PLS_INTEGER := 0;

PROCEDURE Write_Log (p_line VARCHAR2, p_debug_level PLS_INTEGER DEFAULT 0) IS
BEGIN

  IF Utils.g_debug_level >= p_debug_level THEN
    Utils.Write_Log (p_line);
  END IF;

END Write_Log;

FUNCTION Dedup_Hash (p_card PLS_INTEGER, p_key PLS_INTEGER, p_hash int_hash_type) RETURN PLS_INTEGER IS
  l_trial_key       PLS_INTEGER := p_card * p_key;
BEGIN

  LOOP

    IF p_hash.EXISTS (l_trial_key) THEN
      l_trial_key := l_trial_key + 1;
    ELSE
      EXIT;
    END IF;

  END LOOP;
  RETURN l_trial_key;

END Dedup_Hash;

PROCEDURE Pop_Arrays (p_cat_cur SYS_REFCURSOR, p_item_cur SYS_REFCURSOR) IS
  n_cat                     PLS_INTEGER := 0;
  l_price                   PLS_INTEGER;
  l_profit                  PLS_INTEGER;

  l_last_cat                VARCHAR2(30) := '???';

  l_item_price_hash         int_hash_type;
  l_item_profit_hash        int_hash_type;

BEGIN

  FETCH p_cat_cur BULK COLLECT INTO g_cat_list;
  CLOSE p_cat_cur;
  Write_Log ('Collected ' || g_cat_list.COUNT || ' cats');

  FETCH p_item_cur BULK COLLECT INTO g_item_cat_list;
  CLOSE p_item_cur;
  Write_Log ('Collected ' || g_item_cat_list.COUNT || ' items');


  Write_Log (g_n_cats || ' cats');

  g_n_cats := g_cat_list.COUNT - 1;
  g_item_range_list.EXTEND (g_n_cats);
  FOR i IN 1..g_cat_list.COUNT LOOP

    IF g_cat_list(i).id = c_cat_all THEN
      g_set_size := g_cat_list(i).min_items;
    ELSE
      g_cat_hash (g_cat_list(i).id) := i;
    END IF;

  END LOOP;
  g_cat_list.TRIM;

  FOR i IN 1..g_item_cat_list.COUNT LOOP

    IF g_item_cat_list(i).price < g_min_item_price THEN
      g_min_item_price := g_item_cat_list(i).price;
    END IF;

    IF g_item_cat_list(i).profit > g_max_item_profit THEN
      g_max_item_profit := g_item_cat_list(i).profit;
    END IF;
    l_item_price_hash (Dedup_Hash (p_card => g_item_cat_list.COUNT, p_key => g_item_cat_list(i).price, p_hash => l_item_price_hash)) := i;
    l_item_profit_hash (Dedup_Hash (p_card => g_item_cat_list.COUNT, p_key => g_item_cat_list(i).profit, p_hash => l_item_profit_hash)) := i;

    IF g_item_cat_list(i).cat_id != l_last_cat THEN
--
-- Cat has changed, so reset the itm number to zero, and assign the list of items 
--  for previous cat
--
      n_cat := n_cat + 1;
      g_item_range_list (n_cat).item_beg := i;
      IF i > 1 THEN
        g_item_range_list (n_cat - 1).item_end := i - 1;
      END IF;
      l_last_cat := g_item_cat_list(i).cat_id;

    END IF;

  END LOOP;

  g_n_items := g_item_cat_list.COUNT;
  g_item_range_list (g_n_cats).item_end := g_n_items;
  g_min_price_togo.EXTEND (g_set_size);
  g_max_profit_togo.EXTEND (g_set_size);
  l_price := l_item_price_hash.FIRST;
  l_profit := l_item_profit_hash.LAST;
  Write_Log ('Hash first price min / profit max ' || l_price || ' / ' || l_profit);
  g_min_price_togo (g_set_size) := 0;
  g_max_profit_togo (g_set_size) := 0;

  FOR i IN 1..g_set_size - 1 LOOP

    g_min_price_togo (g_set_size - i) := g_min_price_togo (g_set_size - i + 1) + l_price / g_item_cat_list.COUNT;
    g_max_profit_togo (g_set_size - i) := g_max_profit_togo (g_set_size - i + 1) + l_profit / g_item_cat_list.COUNT;

    l_price := l_item_price_hash.NEXT (l_price);
    l_profit := l_item_profit_hash.PRIOR (l_profit);
    Write_Log ((g_set_size - i) || ': price min / profit max ' || g_min_price_togo (g_set_size - i) || ' / ' || g_max_profit_togo (g_set_size - i));

  END LOOP;

  Write_Log ('Price min / profit max ' || g_min_item_price || ' / ' || g_max_item_profit);
  FOR i IN 1..g_n_cats LOOP

    Utils.Write_Log ('Cat ' || i || ' : ' || g_cat_list(i).id || ' - ' || g_cat_list(i).min_items || ' - ' || g_cat_list(i).max_items || ' - ' || g_item_range_list(i).item_beg || ' - ' || g_item_range_list(i).item_end);

  END LOOP;

  g_sol_list.EXTEND (g_n_size);
  FOR i IN 1..g_n_size LOOP
    g_profit_hash (i) := i;
  END LOOP;
  g_nth_profit := g_profit_hash.FIRST;
  g_trial_sol.price := 0;
  g_trial_sol.profit := 0;

END Pop_Arrays;

PROCEDURE Get_Best_Item_List (p_position PLS_INTEGER, p_item_index_beg PLS_INTEGER, p_item_index_end PLS_INTEGER, x_item_hash IN OUT NOCOPY int_hash_type) IS

PROCEDURE Check_Item (p_item_index PLS_INTEGER) IS

  l_item_rec         item_cat_rec_type := g_item_cat_list (p_item_index);
  l_item_list        num_list_type;
  Item_Failed        EXCEPTION;
  l_item_str         VARCHAR2(200) := LPad (l_item_rec.id, (p_position)*3, '.') || '-' || l_item_rec.cat_id  || '-' || l_item_rec.price  || '-' || l_item_rec.profit;

  FUNCTION Price_LB (p_position PLS_INTEGER) RETURN PLS_INTEGER IS
  BEGIN

    RETURN g_min_price_togo (p_position);
      
  END Price_LB;

  FUNCTION Profit_UB (p_position PLS_INTEGER) RETURN PLS_INTEGER IS
  BEGIN

    RETURN g_max_profit_togo (p_position);

  END Profit_UB;

BEGIN

  IF l_item_rec.price + g_trial_sol.price + Price_LB (p_position) > g_max_price THEN

    l_item_str := l_item_str || ' [price failed ' || (l_item_rec.price + g_trial_sol.price) || ']';
    IF (g_set_size - p_position) = 0 THEN
      Write_Log ('Solution fails with price of ' || (l_item_rec.price + g_trial_sol.price), 1);
    END IF;
    RAISE Item_Failed;
  END IF;

  IF l_item_rec.profit + g_trial_sol.profit + Profit_UB (p_position) <= g_nth_profit THEN
    l_item_str := l_item_str || ' [profit failed ' || (l_item_rec.profit + g_trial_sol.profit) || ', nth = ' || g_nth_profit || ']';
    IF (g_set_size - p_position) = 0 THEN
      Write_Log ('Solution fails with profit of ' || (l_item_rec.profit + g_trial_sol.profit), 1);
      g_n_sols := g_n_sols + 1;
    END IF;
    RAISE Item_Failed;
  END IF;

  x_item_hash (Dedup_Hash (p_card => g_keep_size, p_key => l_item_rec.profit + g_trial_sol.profit, p_hash => x_item_hash)) := p_item_index;

EXCEPTION

  WHEN Item_Failed THEN
    Write_Log (l_item_str, 2);
    
END Check_Item;

BEGIN

  FOR i IN p_item_index_beg..p_item_index_end LOOP

    Check_Item (i);

  END LOOP;

END Get_Best_Item_List;

PROCEDURE Add_Solution IS
  l_nth_index        PLS_INTEGER;
BEGIN

  g_n_sols := g_n_sols + 1;
  l_nth_index := g_profit_hash (g_profit_hash.FIRST);
  Write_Log ('Solution replaces in position ' || l_nth_index || ' profit is ' || g_trial_sol.profit || ' price is ' || g_trial_sol.price, 1);

  g_profit_hash.DELETE (g_profit_hash.FIRST);
  g_profit_hash (Dedup_Hash (p_card => g_n_size, p_key => g_trial_sol.profit, p_hash => g_profit_hash)) := l_nth_index;

  g_sol_list (l_nth_index) := g_trial_sol;
  g_nth_profit := g_profit_hash.FIRST / g_n_size;

  IF Mod (g_n_sols, c_hash_renew_point) = 0 THEN -- Not sur eif this works, but is intended to clear memory overhang
    g_hash_buffer :=  g_profit_hash;
    g_profit_hash :=  g_hash_buffer;
  END IF;

END Add_Solution;

PROCEDURE Add_Item_To_Trial (p_position PLS_INTEGER, p_item_index PLS_INTEGER) IS

  l_item_rec          item_cat_rec_type := g_item_cat_list (p_item_index);

BEGIN

  g_trial_sol.price := g_trial_sol.price + l_item_rec.price;
  g_trial_sol.profit := g_trial_sol.profit + l_item_rec.profit;

  IF g_trial_sol.item_list IS NULL THEN
    g_trial_sol.item_list := id_list_type (l_item_rec.id);
  ELSE
    g_trial_sol.item_list.EXTEND;
    g_trial_sol.item_list (p_position) := l_item_rec.id;
  END IF;

  IF p_position = g_set_size THEN

    Add_Solution;

  END IF;
   
END Add_Item_To_Trial;

FUNCTION Try_Position (p_position PLS_INTEGER, p_n_curr_cat PLS_INTEGER, p_cat_index_beg PLS_INTEGER, p_item_index_beg PLS_INTEGER) RETURN BOOLEAN IS

  l_item_hash       int_hash_type;
  l_item_index      PLS_INTEGER;
  l_cat_index_beg   PLS_INTEGER := p_cat_index_beg;
  l_item_index_beg  PLS_INTEGER := p_item_index_beg;
  l_n_curr_cat      PLS_INTEGER := p_n_curr_cat;
  l_profit          PLS_INTEGER;
BEGIN

  g_n_recursive_calls := g_n_recursive_calls + 1;
  IF g_n_recursive_calls > g_max_calls THEN
    Write_Log (LPad ('*', p_position, '*') || 'Truncating search after ' || g_max_calls || ' recursive calls***');
    RETURN TRUE;
  END IF;

  IF p_n_curr_cat = g_cat_list (p_cat_index_beg).max_items THEN
-- 
-- passed in the cat we were on in last position
-- check max not passed, if so go to next cat and reset item range
--
    Write_Log ('Maxed Cat ' || p_cat_index_beg || ': ' || p_n_curr_cat || '-' || g_cat_list (p_cat_index_beg).max_items, 5);
    l_cat_index_beg := p_cat_index_beg + 1;
    IF l_cat_index_beg > g_n_cats THEN
      RETURN FALSE;
    END IF;
    l_item_index_beg := g_item_range_list (l_cat_index_beg).item_beg;
    l_n_curr_cat := 0;

  END IF;

  FOR j IN l_cat_index_beg..g_n_cats LOOP

    IF l_item_index_beg < g_item_range_list (j).item_beg THEN
      l_item_index_beg := g_item_range_list (j).item_beg;
    END IF;

    Write_Log ('Start Cat ' || j || ': ' || l_n_curr_cat || '-' || g_cat_list(j).min_items, 5);
    l_n_curr_cat := l_n_curr_cat + 1;
    l_item_hash.DELETE;
    Get_Best_Item_List (p_position => p_position, p_item_index_beg => l_item_index_beg, p_item_index_end => g_item_range_list(j).item_end, x_item_hash => l_item_hash);
    IF l_item_hash IS NOT NULL THEN

    l_profit := l_item_hash.LAST;
    FOR i IN 1..Least (g_keep_size, l_item_hash.COUNT) LOOP
      l_item_index := l_item_hash (l_profit);
      Write_Log (LPad (g_item_cat_list (l_item_index).id, (p_position)*3, '.') || '-' || g_item_cat_list (l_item_index).cat_id  || '-' || g_item_cat_list (l_item_index).price  || '-' || g_item_cat_list (l_item_index).profit, 1);
      Add_Item_To_Trial (p_position => p_position, p_item_index => l_item_index);
      IF p_position < g_set_size THEN

        IF Try_Position (p_position => p_position + 1, p_n_curr_cat => l_n_curr_cat, p_cat_index_beg => j, p_item_index_beg => l_item_index + 1) THEN RETURN TRUE; END IF;

      END IF;

      IF g_trial_sol.item_list IS NOT NULL AND g_trial_sol.item_list.COUNT = p_position THEN
        g_trial_sol.item_list.TRIM;
        g_trial_sol.price := g_trial_sol.price - g_item_cat_list (l_item_index).price;
        g_trial_sol.profit := g_trial_sol.profit - g_item_cat_list (l_item_index).profit;
      END IF;

      l_profit := l_item_hash.PRIOR (l_profit);

    END LOOP;

    ELSE
      Write_Log ('No items found');
    END IF;
-- 
--  Don't look at any more cats if we are not past the minimum for the current one at this position
--
    Write_Log ('Cat ' || j || ': ' || l_n_curr_cat || '-' || g_cat_list(j).min_items, 5);
    IF l_n_curr_cat <= g_cat_list(j).min_items THEN
      EXIT;
    END IF;

    l_n_curr_cat := 0;

  END LOOP;
  RETURN FALSE;

END Try_Position;

FUNCTION Best_N_Sets (  p_keep_size     PLS_INTEGER, 
                        p_max_calls     PLS_INTEGER, 
                        p_n_size        PLS_INTEGER, 
                        p_max_price     PLS_INTEGER, 
                        p_cat_cur       SYS_REFCURSOR, 
                        p_item_cur      SYS_REFCURSOR) RETURN sol_detail_list_type PIPELINED IS

  l_sol_detail_rec          sol_detail_rec_type;
  l_position                PLS_INTEGER := 1;
BEGIN

  g_keep_size := p_keep_size; g_max_calls := p_max_calls; g_n_size := p_n_size; g_max_price := p_max_price;

  Pop_Arrays (p_cat_cur, p_item_cur);

  IF Try_Position (p_position => 1, p_n_curr_cat => 0, p_cat_index_beg => 1, p_item_index_beg => 1) THEN NULL; END IF;

  FOR i IN 1..g_n_size LOOP

    l_sol_detail_rec.set_id := i;
    l_sol_detail_rec.sol_price := g_sol_list(i).price;
    l_sol_detail_rec.sol_profit := g_sol_list(i).profit;

    IF g_sol_list(i).item_list IS NOT NULL THEN

      FOR j IN 1..g_sol_list(i).item_list.COUNT LOOP

        l_sol_detail_rec.item_id := g_sol_list(i).item_list(j);
        PIPE ROW (l_sol_detail_rec);

      END LOOP;

    END IF;

  END LOOP;

  Write_Log (g_n_sols || ' solutions found in ' || g_n_recursive_calls || ' recursive calls');

  RETURN;

END Best_N_Sets;

END Item_Cats;
/
SHO ERR

How It Works

The solution approach uses a modified depth-first recursion, following a similar idea to the SQL method, of adding items in strictly increasing order of category and profit ranking. Treatment of constraints uses similar ideas to the SQL solution.

  • The package is completely generic, with the items and categories being specified by means of input cursors
  • Depth-first is modified by a ranking of the next sets of feasible items, partitioned by category, in order to limit the number progressed
  • Hashes (associative arrays in Oracle) are used for ranking
  • A function, Dedup_Hash is used to allow for duplicate hash keys; it works by storing as key the actual key multiplied by the ranking set cardinality, then adding one iteratively until no duplication occurs
  • The recursion is truncated if the number of recursive calls exceeds an input limit
  • The input cursors are read into arrays and all subsequent processing is in memory; this is not a scalability problem because only the current best solutions are retained; also, I reset hashes after a given number of updates

Results

Test Problem 1: Brazilian League

The pipelined function solved this in 5 seconds, while the SQL solution solved it in 21 seconds. The solutions were identical, as follows:

TOT_PROFIT  TOT_PRICE        RNK PO P_ID PLAYER_NAME                    CLUB_NAME                           PRICE AVG_POINTS
---------- ---------- ---------- -- ---- ------------------------------ ------------------------------ ---------- ----------
     10923      18176          1 CB 098  Digão                          Fluminense                            931        927
                                    099  Samir                          Flamengo                              267        680
                                 CO 078  Jaime De AlMFda                Flamengo                             1156        803
                                 FW 001  Éderson                        Atlético-PR                          1712       1012
                                    002  Maxi Biancucchi                Vitória                              1962       1005
                                    003  Rafael Sobis                   Fluminense                           2303        955
                                 GK 022  Wilson                         Vitória                              1239        794
                                 MF 058  Fred                           Internacional                        3028        892
                                    059  Zé Roberto                     Grêmio                               2593        878
                                    060  Otavinho                       Internacional                         762        807
                                 WB 038  Ivan                           Portuguesa                            755       1320
                                    039  Elsinho                        Vasco                                1468        850
                19027          2 CB 098  Digão                          Fluminense                            931        927
                                    099  Samir                          Flamengo                              267        680
                                 CO 078  Jaime De AlMFda                Flamengo                             1156        803
                                 FW 001  Éderson                        Atlético-PR                          1712       1012
                                    002  Maxi Biancucchi                Vitória                              1962       1005
                                    003  Rafael Sobis                   Fluminense                           2303        955
                                 GK 021  Fábio                          Cruzeiro                             2090        794
                                 MF 058  Fred                           Internacional                        3028        892
                                    059  Zé Roberto                     Grêmio                               2593        878
                                    060  Otavinho                       Internacional                         762        807
                                 WB 038  Ivan                           Portuguesa                            755       1320
                                    039  Elsinho                        Vasco                                1468        850
     10905      18795          3 CB 098  Digão                          Fluminense                            931        927
                                    099  Samir                          Flamengo                              267        680
                                 CO 078  Jaime De AlMFda                Flamengo                             1156        803
                                 FW 001  Éderson                        Atlético-PR                          1712       1012
                                    002  Maxi Biancucchi                Vitória                              1962       1005
                                    003  Rafael Sobis                   Fluminense                           2303        955
                                 GK 023  Vanderlei                      Coritiba                             1858        776
                                 MF 058  Fred                           Internacional                        3028        892
                                    059  Zé Roberto                     Grêmio                               2593        878
                                    060  Otavinho                       Internacional                         762        807
                                 WB 038  Ivan                           Portuguesa                            755       1320
                                    039  Elsinho                        Vasco                                1468        850
     10833      18994          4 CB 098  Digão                          Fluminense                            931        927
                                    102  Bressan                        Grêmio                               1085        590
                                 CO 078  Jaime De AlMFda                Flamengo                             1156        803
                                 FW 001  Éderson                        Atlético-PR                          1712       1012
                                    002  Maxi Biancucchi                Vitória                              1962       1005
                                    003  Rafael Sobis                   Fluminense                           2303        955
                                 GK 022  Wilson                         Vitória                              1239        794
                                 MF 058  Fred                           Internacional                        3028        892
                                    059  Zé Roberto                     Grêmio                               2593        878
                                    060  Otavinho                       Internacional                         762        807
                                 WB 038  Ivan                           Portuguesa                            755       1320
                                    039  Elsinho                        Vasco                                1468        850
                19845          5 CB 098  Digão                          Fluminense                            931        927
                                    102  Bressan                        Grêmio                               1085        590
                                 CO 078  Jaime De AlMFda                Flamengo                             1156        803
                                 FW 001  Éderson                        Atlético-PR                          1712       1012
                                    002  Maxi Biancucchi                Vitória                              1962       1005
                                    003  Rafael Sobis                   Fluminense                           2303        955
                                 GK 021  Fábio                          Cruzeiro                             2090        794
                                 MF 058  Fred                           Internacional                        3028        892
                                    059  Zé Roberto                     Grêmio                               2593        878
                                    060  Otavinho                       Internacional                         762        807
                                 WB 038  Ivan                           Portuguesa                            755       1320
                                    039  Elsinho                        Vasco                                1468        850
     10831      19608          6 CB 098  Digão                          Fluminense                            931        927
                                    103  Manoel                         Atlético-PR                          1699        588
                                 CO 078  Jaime De AlMFda                Flamengo                             1156        803
                                 FW 001  Éderson                        Atlético-PR                          1712       1012
                                    002  Maxi Biancucchi                Vitória                              1962       1005
                                    003  Rafael Sobis                   Fluminense                           2303        955
                                 GK 022  Wilson                         Vitória                              1239        794
                                 MF 058  Fred                           Internacional                        3028        892
                                    059  Zé Roberto                     Grêmio                               2593        878
                                    060  Otavinho                       Internacional                         762        807
                                 WB 038  Ivan                           Portuguesa                            755       1320
                                    039  Elsinho                        Vasco                                1468        850
     10825      18190          7 CB 098  Digão                          Fluminense                            931        927
                                    099  Samir                          Flamengo                              267        680
                                 CO 078  Jaime De AlMFda                Flamengo                             1156        803
                                 FW 001  Éderson                        Atlético-PR                          1712       1012
                                    002  Maxi Biancucchi                Vitória                              1962       1005
                                    003  Rafael Sobis                   Fluminense                           2303        955
                                 GK 022  Wilson                         Vitória                              1239        794
                                 MF 058  Fred                           Internacional                        3028        892
                                    059  Zé Roberto                     Grêmio                               2593        878
                                    060  Otavinho                       Internacional                         762        807
                                 WB 038  Ivan                           Portuguesa                            755       1320
                                    040  Egídio                         Cruzeiro                             1482        752
                19041          8 CB 098  Digão                          Fluminense                            931        927
                                    099  Samir                          Flamengo                              267        680
                                 CO 078  Jaime De AlMFda                Flamengo                             1156        803
                                 FW 001  Éderson                        Atlético-PR                          1712       1012
                                    002  Maxi Biancucchi                Vitória                              1962       1005
                                    003  Rafael Sobis                   Fluminense                           2303        955
                                 GK 021  Fábio                          Cruzeiro                             2090        794
                                 MF 058  Fred                           Internacional                        3028        892
                                    059  Zé Roberto                     Grêmio                               2593        878
                                    060  Otavinho                       Internacional                         762        807
                                 WB 038  Ivan                           Portuguesa                            755       1320
                                    040  Egídio                         Cruzeiro                             1482        752
     10821      19370          9 CB 098  Digão                          Fluminense                            931        927
                                    104  Cléber                         Ponte Preta                          1461        578
                                 CO 078  Jaime De AlMFda                Flamengo                             1156        803
                                 FW 001  Éderson                        Atlético-PR                          1712       1012
                                    002  Maxi Biancucchi                Vitória                              1962       1005
                                    003  Rafael Sobis                   Fluminense                           2303        955
                                 GK 022  Wilson                         Vitória                              1239        794
                                 MF 058  Fred                           Internacional                        3028        892
                                    059  Zé Roberto                     Grêmio                               2593        878
                                    060  Otavinho                       Internacional                         762        807
                                 WB 038  Ivan                           Portuguesa                            755       1320
                                    039  Elsinho                        Vasco                                1468        850
     10815      19613         10 CB 098  Digão                          Fluminense                            931        927
                                    102  Bressan                        Grêmio                               1085        590
                                 CO 078  Jaime De AlMFda                Flamengo                             1156        803
                                 FW 001  Éderson                        Atlético-PR                          1712       1012
                                    002  Maxi Biancucchi                Vitória                              1962       1005
                                    003  Rafael Sobis                   Fluminense                           2303        955
                                 GK 023  Vanderlei                      Coritiba                             1858        776
                                 MF 058  Fred                           Internacional                        3028        892
                                    059  Zé Roberto                     Grêmio                               2593        878
                                    060  Otavinho                       Internacional                         762        807
                                 WB 038  Ivan                           Portuguesa                            755       1320
                                    039  Elsinho                        Vasco                                1468        850

120 rows selected.

Test Problem 2: English Premier League

I used a maximum price of 900, and a keep parameter of 40, meaning retain the best 40 records by partition during recursion for the SQL and a value of 10 for the pipelined function. The keep parameter operates differently in the two cases so does not need to be the same value.

The SQL solution took 98 seconds, while the pipelined function took 290 seconds. Both methods got the same best solution, but the tenth best was marginally better for the SQL, at 1965, compared with 1962 for pipelined function (which truncated after 100000 recursive calls).

[For followers of Manchester United, and David Moyes, it may be of interest to note that all of the best solutions included both Leighton Baines and Patrice Evra, and the best also had Marouane Fellaini. Will these players be united also in the real world next season?]

SQL Solution

TOT_PROFIT  TOT_PRICE        RNK PO P_ID PLAYER_NAME                    CLUB_NAME            PRICE AVG_POINTS
---------- ---------- ---------- -- ---- ------------------------------ --------------- ---------- ----------
      1973        898          1 DF 165  Leighton Baines                Everton                 78        173
                                    332  Patrice Evra                   Man Utd                 73        152
                                    268  Glen Johnson                   Liverpool               65        141
                                 FW 286  Luis Suarez                    Liverpool              105        213
                                    533  Rickie Lambert                 Southampton             69        178
                                 GK 549  Asmir Begovic                  Stoke City              56        154
                                 MF 661  Gareth Bale                    Tottenham              111        240
                                    030  Santi Santi Cazorla            Arsenal                 97        198
                                    265  Steven Gerrard                 Liverpool               92        187
                                    641  Miguel Michu                   Swansea                 79        169
                                    177  Marouane Fellaini              Everton                 73        168
      1971        899          2 DF 165  Leighton Baines                Everton                 78        173
                                    332  Patrice Evra                   Man Utd                 73        152
                                    268  Glen Johnson                   Liverpool               65        141
                                 FW 286  Luis Suarez                    Liverpool              105        213
                                    533  Rickie Lambert                 Southampton             69        178
                                    047  Christian Benteke              Aston Villa             74        166
                                 GK 549  Asmir Begovic                  Stoke City              56        154
                                 MF 661  Gareth Bale                    Tottenham              111        240
                                    030  Santi Santi Cazorla            Arsenal                 97        198
                                    265  Steven Gerrard                 Liverpool               92        187
                                    641  Miguel Michu                   Swansea                 79        169
      1970        893          3 DF 165  Leighton Baines                Everton                 78        173
                                    332  Patrice Evra                   Man Utd                 73        152
                                    268  Glen Johnson                   Liverpool               65        141
                                 FW 286  Luis Suarez                    Liverpool              105        213
                                    533  Rickie Lambert                 Southampton             69        178
                                    047  Christian Benteke              Aston Villa             74        166
                                 GK 549  Asmir Begovic                  Stoke City              56        154
                                 MF 661  Gareth Bale                    Tottenham              111        240
                                    030  Santi Santi Cazorla            Arsenal                 97        198
                                    265  Steven Gerrard                 Liverpool               92        187
                                    177  Marouane Fellaini              Everton                 73        168
      1968        899          4 DF 165  Leighton Baines                Everton                 78        173
                                    332  Patrice Evra                   Man Utd                 73        152
                                    268  Glen Johnson                   Liverpool               65        141
                                 FW 286  Luis Suarez                    Liverpool              105        213
                                    533  Rickie Lambert                 Southampton             69        178
                                    047  Christian Benteke              Aston Villa             74        166
                                 MF 661  Gareth Bale                    Tottenham              111        240
                                    030  Santi Santi Cazorla            Arsenal                 97        198
                                    265  Steven Gerrard                 Liverpool               92        187
                                    177  Marouane Fellaini              Everton                 73        168
                                    428  Robert Snodgrass               Norwich                 62        152
                               5 DF 165  Leighton Baines                Everton                 78        173
                                    332  Patrice Evra                   Man Utd                 73        152
                                    569  Ryan Shawcross                 Stoke City              56        133
                                 FW 286  Luis Suarez                    Liverpool              105        213
                                    533  Rickie Lambert                 Southampton             69        178
                                 GK 549  Asmir Begovic                  Stoke City              56        154
                                 MF 661  Gareth Bale                    Tottenham              111        240
                                    030  Santi Santi Cazorla            Arsenal                 97        198
                                    149  Juan Mata                      Chelsea                102        190
                                    641  Miguel Michu                   Swansea                 79        169
                                    177  Marouane Fellaini              Everton                 73        168
                  900          6 DF 165  Leighton Baines                Everton                 78        173
                                    332  Patrice Evra                   Man Utd                 73        152
                                    268  Glen Johnson                   Liverpool               65        141
                                 FW 286  Luis Suarez                    Liverpool              105        213
                                    533  Rickie Lambert                 Southampton             69        178
                                    204  Dimitar Berbatov               Fulham                  71        161
                                 GK 549  Asmir Begovic                  Stoke City              56        154
                                 MF 661  Gareth Bale                    Tottenham              111        240
                                    030  Santi Santi Cazorla            Arsenal                 97        198
                                    149  Juan Mata                      Chelsea                102        190
                                    177  Marouane Fellaini              Everton                 73        168
      1966        896          7 DF 165  Leighton Baines                Everton                 78        173
                                    332  Patrice Evra                   Man Utd                 73        152
                                    268  Glen Johnson                   Liverpool               65        141
                                 FW 286  Luis Suarez                    Liverpool              105        213
                                    533  Rickie Lambert                 Southampton             69        178
                                    204  Dimitar Berbatov               Fulham                  71        161
                                 GK 549  Asmir Begovic                  Stoke City              56        154
                                 MF 661  Gareth Bale                    Tottenham              111        240
                                    030  Santi Santi Cazorla            Arsenal                 97        198
                                    265  Steven Gerrard                 Liverpool               92        187
                                    641  Miguel Michu                   Swansea                 79        169
                  900          8 DF 165  Leighton Baines                Everton                 78        173
                                    332  Patrice Evra                   Man Utd                 73        152
                                    569  Ryan Shawcross                 Stoke City              56        133
                                 FW 286  Luis Suarez                    Liverpool              105        213
                                    533  Rickie Lambert                 Southampton             69        178
                                    047  Christian Benteke              Aston Villa             74        166
                                 GK 549  Asmir Begovic                  Stoke City              56        154
                                 MF 661  Gareth Bale                    Tottenham              111        240
                                    030  Santi Santi Cazorla            Arsenal                 97        198
                                    149  Juan Mata                      Chelsea                102        190
                                    641  Miguel Michu                   Swansea                 79        169
      1965        890          9 DF 165  Leighton Baines                Everton                 78        173
                                    332  Patrice Evra                   Man Utd                 73        152
                                    268  Glen Johnson                   Liverpool               65        141
                                 FW 286  Luis Suarez                    Liverpool              105        213
                                    533  Rickie Lambert                 Southampton             69        178
                                    204  Dimitar Berbatov               Fulham                  71        161
                                 GK 549  Asmir Begovic                  Stoke City              56        154
                                 MF 661  Gareth Bale                    Tottenham              111        240
                                    030  Santi Santi Cazorla            Arsenal                 97        198
                                    265  Steven Gerrard                 Liverpool               92        187
                                    177  Marouane Fellaini              Everton                 73        168
                  894         10 DF 165  Leighton Baines                Everton                 78        173
                                    332  Patrice Evra                   Man Utd                 73        152
                                    569  Ryan Shawcross                 Stoke City              56        133
                                 FW 286  Luis Suarez                    Liverpool              105        213
                                    533  Rickie Lambert                 Southampton             69        178
                                    047  Christian Benteke              Aston Villa             74        166
                                 GK 549  Asmir Begovic                  Stoke City              56        154
                                 MF 661  Gareth Bale                    Tottenham              111        240
                                    030  Santi Santi Cazorla            Arsenal                 97        198
                                    149  Juan Mata                      Chelsea                102        190
                                    177  Marouane Fellaini              Everton                 73        168

110 rows selected.

Elapsed: 00:01:38.26

Pipelined Function Solution

SOL_PROFIT  SOL_PRICE        RNK PO ITEM_ID    CLUB_NAME       PLAYER_NAME               PRICE AVG_POINTS
---------- ---------- ---------- -- ---------- --------------- -------------------- ---------- ----------
      1973        898          1 DF 165        Everton         Leighton Baines              78        173
                                    268        Liverpool       Glen Johnson                 65        141
                                    332        Man Utd         Patrice Evra                 73        152
                                 FW 286        Liverpool       Luis Suarez                 105        213
                                    533        Southampton     Rickie Lambert               69        178
                                 GK 549        Stoke City      Asmir Begovic                56        154
                                 MF 177        Everton         Marouane Fellaini            73        168
                                    265        Liverpool       Steven Gerrard               92        187
                                    30         Arsenal         Santi Santi Cazorla          97        198
                                    641        Swansea         Miguel Michu                 79        169
                                    661        Tottenham       Gareth Bale                 111        240
      1971        899          2 DF 165        Everton         Leighton Baines              78        173
                                    268        Liverpool       Glen Johnson                 65        141
                                    332        Man Utd         Patrice Evra                 73        152
                                 FW 286        Liverpool       Luis Suarez                 105        213
                                    47         Aston Villa     Christian Benteke            74        166
                                    533        Southampton     Rickie Lambert               69        178
                                 GK 549        Stoke City      Asmir Begovic                56        154
                                 MF 265        Liverpool       Steven Gerrard               92        187
                                    30         Arsenal         Santi Santi Cazorla          97        198
                                    641        Swansea         Miguel Michu                 79        169
                                    661        Tottenham       Gareth Bale                 111        240
      1970        893          3 DF 165        Everton         Leighton Baines              78        173
                                    268        Liverpool       Glen Johnson                 65        141
                                    332        Man Utd         Patrice Evra                 73        152
                                 FW 286        Liverpool       Luis Suarez                 105        213
                                    47         Aston Villa     Christian Benteke            74        166
                                    533        Southampton     Rickie Lambert               69        178
                                 GK 549        Stoke City      Asmir Begovic                56        154
                                 MF 177        Everton         Marouane Fellaini            73        168
                                    265        Liverpool       Steven Gerrard               92        187
                                    30         Arsenal         Santi Santi Cazorla          97        198
                                    661        Tottenham       Gareth Bale                 111        240
      1968        900          4 DF 165        Everton         Leighton Baines              78        173
                                    268        Liverpool       Glen Johnson                 65        141
                                    332        Man Utd         Patrice Evra                 73        152
                                 FW 204        Fulham          Dimitar Berbatov             71        161
                                    286        Liverpool       Luis Suarez                 105        213
                                    533        Southampton     Rickie Lambert               69        178
                                 GK 549        Stoke City      Asmir Begovic                56        154
                                 MF 149        Chelsea         Juan Mata                   102        190
                                    177        Everton         Marouane Fellaini            73        168
                                    30         Arsenal         Santi Santi Cazorla          97        198
                                    661        Tottenham       Gareth Bale                 111        240
      1966        896          5 DF 165        Everton         Leighton Baines              78        173
                                    268        Liverpool       Glen Johnson                 65        141
                                    332        Man Utd         Patrice Evra                 73        152
                                 FW 204        Fulham          Dimitar Berbatov             71        161
                                    286        Liverpool       Luis Suarez                 105        213
                                    533        Southampton     Rickie Lambert               69        178
                                 GK 549        Stoke City      Asmir Begovic                56        154
                                 MF 265        Liverpool       Steven Gerrard               92        187
                                    30         Arsenal         Santi Santi Cazorla          97        198
                                    641        Swansea         Miguel Michu                 79        169
                                    661        Tottenham       Gareth Bale                 111        240
      1965        890          6 DF 165        Everton         Leighton Baines              78        173
                                    268        Liverpool       Glen Johnson                 65        141
                                    332        Man Utd         Patrice Evra                 73        152
                                 FW 204        Fulham          Dimitar Berbatov             71        161
                                    286        Liverpool       Luis Suarez                 105        213
                                    533        Southampton     Rickie Lambert               69        178
                                 GK 549        Stoke City      Asmir Begovic                56        154
                                 MF 177        Everton         Marouane Fellaini            73        168
                                    265        Liverpool       Steven Gerrard               92        187
                                    30         Arsenal         Santi Santi Cazorla          97        198
                                    661        Tottenham       Gareth Bale                 111        240
                  897          7 DF 165        Everton         Leighton Baines              78        173
                                    248        Liverpool       Daniel Agger                 64        133
                                    332        Man Utd         Patrice Evra                 73        152
                                 FW 286        Liverpool       Luis Suarez                 105        213
                                    533        Southampton     Rickie Lambert               69        178
                                 GK 549        Stoke City      Asmir Begovic                56        154
                                 MF 177        Everton         Marouane Fellaini            73        168
                                    265        Liverpool       Steven Gerrard               92        187
                                    30         Arsenal         Santi Santi Cazorla          97        198
                                    641        Swansea         Miguel Michu                 79        169
                                    661        Tottenham       Gareth Bale                 111        240
      1964        895          8 DF 165        Everton         Leighton Baines              78        173
                                    268        Liverpool       Glen Johnson                 65        141
                                    332        Man Utd         Patrice Evra                 73        152
                                 FW 286        Liverpool       Luis Suarez                 105        213
                                    533        Southampton     Rickie Lambert               69        178
                                    720        West Brom       Romelu Lukaku                66        157
                                 GK 549        Stoke City      Asmir Begovic                56        154
                                 MF 149        Chelsea         Juan Mata                   102        190
                                    177        Everton         Marouane Fellaini            73        168
                                    30         Arsenal         Santi Santi Cazorla          97        198
                                    661        Tottenham       Gareth Bale                 111        240
      1963        898          9 DF 165        Everton         Leighton Baines              78        173
                                    248        Liverpool       Daniel Agger                 64        133
                                    332        Man Utd         Patrice Evra                 73        152
                                 FW 286        Liverpool       Luis Suarez                 105        213
                                    47         Aston Villa     Christian Benteke            74        166
                                    533        Southampton     Rickie Lambert               69        178
                                 GK 549        Stoke City      Asmir Begovic                56        154
                                 MF 265        Liverpool       Steven Gerrard               92        187
                                    30         Arsenal         Santi Santi Cazorla          97        198
                                    641        Swansea         Miguel Michu                 79        169
                                    661        Tottenham       Gareth Bale                 111        240
      1962        891         10 DF 165        Everton         Leighton Baines              78        173
                                    268        Liverpool       Glen Johnson                 65        141
                                    332        Man Utd         Patrice Evra                 73        152
                                 FW 286        Liverpool       Luis Suarez                 105        213
                                    533        Southampton     Rickie Lambert               69        178
                                    720        West Brom       Romelu Lukaku                66        157
                                 GK 549        Stoke City      Asmir Begovic                56        154
                                 MF 265        Liverpool       Steven Gerrard               92        187
                                    30         Arsenal         Santi Santi Cazorla          97        198
                                    641        Swansea         Miguel Michu                 79        169
                                    661        Tottenham       Gareth Bale                 111        240

110 rows selected.

Elapsed: 00:04:49.92

Conclusions

My idea for using recursive subquery factoring to solve combinatorial optimisation problems, such as knapsack problems, described in other articles on my blog, was previously only practical for small problems. The extensions described here render it a practical proposition even for larger problems. It is also relatively simple compared with procedural approaches.






SQL for the Balanced Number Partitioning Problem

I noticed a post on AskTom recently that referred to an SQL solution to a version of the so-called Bin Fitting problem, where even distribution is the aim. The solution, How do I solve a Bin Fitting problem in an SQL statement?, uses Oracle's Model clause, and, as the poster of the link observed, has the drawback that the number of bins is embedded in the query structure. I thought it might be interesting to find solutions without that drawback, so that the number of bins could be passed to the query as a bind variable. I came up with three solutions using different techniques, starting here.

An interesting article in American Scientist, The Easiest Hard Problem, notes that the problem is NP-complete, or certifiably hard, but that simple greedy heuristics often produce a good solution, including one used by schoolboys to pick football teams. The article uses the more descriptive term for the problem of balanced number partitioning, and notes some practical applications. The Model clause solution implements a multiple-bin version of the main Greedy Algorithm, while my non-Model SQL solutions implement variants of it that allow other techniques to be used, one of which is very simple and fast: this implements the team picking heuristic for multiple teams.

Another poster, Stew Ashton, suggested a simple change to my Model solution that improved performance, and I use this modified version here. He also suggested that using PL/SQL might be faster, and I have added my own simple PL/SQL implementation of the Greedy Algorithm, as well as a second version of the recursive subquery factoring solution that performs better than the first.

This article explains the solutions, considers two simple examples to illustrate them, and reports on performance testing across dimensions of number of items and number of bins. These show that the solutions exhibit either linear or quadratic variation in execution time with number of items, and some methods are sensitive to the number of bins while others are not.

After I had posted my solutions on the AskTom thread, I came across a thread on OTN, need help to resolve this issue, that requested a solution to a form of bin fitting problem where the bins have fixed capacity and the number of bins required must be determined. I realised that my solutions could easily be extended to add that feature, and posted extended versions of two of the solutions there. I have added a section here for this.

Updated, 5 June 2013: added Model and RSF diagrams

Greedy Algorithm Variants

Say there are N bins and M items.

Greedy Algorithm (GDY)
Set bin sizes zero
Loop over items in descending order of size

  • Add item to current smallest bin
  • Calculate new bin size

End Loop

Greedy Algorithm with Batched Rebalancing (GBR)
Set bin sizes zero
Loop over items in descending order of size in batches of N items

  • Assign batch to N bins, with bins in ascending order of size
  • Calculate new bin sizes

End Loop

Greedy Algorithm with No Rebalancing - or, Team Picking Algorithm (TPA)
Assign items to bins cyclically by bin sequence in descending order of item size

Two Examples

Example: Four Items
Binfit, v1.3 - 4-items
Here we see that the Greedy Algorithm finds the perfect solution, with no difference in bin size, but the two variants have a difference of two.
Example: Six Items
Binfit, v1.3 - 6-items
Here we see that none of the algorithms finds the perfect solution. Both the standard Greedy Algorithm and its batched variant give a difference of two, while the variant without rebalancing gives a difference of four.

SQL Solutions

Original Model for GDY
See the link above for the SQL for the problem with three bins only.

The author has two measures for each bin and implements the GDY algorithm using CASE expressions and aggregation within the rules. The idea is to iterate over the items in descending order of size, setting the item bin to the bin with current smallest value. I use the word 'bin' for his 'bucket'. Some notes:

  • Dimension by row number, ordered by item value
  • Add measures for the iteration, it, and number of iterations required, counter
  • Add measures for the bin name, bucket_name, and current minimum bin value, min_tmp (only first entry used)
  • Add measures for each item bin value, bucket_1-3, being the item value if it's in that bin, else zero
  • Add measures for each bin running sum, pbucket_1-3, being the current value of each bin (only first two entries used)
  • The current minimum bin value, bin_tmp[1] is computed as the least of the running sums
  • The current item bin value is set to the item value for the bin whose value matches the minimum just computed, and null for the others
  • The current bin name is set similarly to be the bin matching the minimum
  • The new running sums are computed for each bin

Brendan's Generic Model for GDY

SELECT item_name, bin, item_value, Max (bin_value) OVER (PARTITION BY bin) bin_value
  FROM (
SELECT * FROM items
  MODEL 
    DIMENSION BY (Row_Number() OVER (ORDER BY item_value DESC) rn)
    MEASURES (item_name, 
              item_value,
              Row_Number() OVER (ORDER BY item_value DESC) bin,
              item_value bin_value,
              Row_Number() OVER (ORDER BY item_value DESC) rn_m,
              0 min_bin,
              Count(*) OVER () - :N_BINS - 1 n_iters
    )
    RULES ITERATE(100000) UNTIL (ITERATION_NUMBER >= n_iters[1]) (
      min_bin[1] = Min(rn_m) KEEP (DENSE_RANK FIRST ORDER BY bin_value)[rn <= :N_BINS],
      bin[ITERATION_NUMBER + :N_BINS + 1] = min_bin[1],
      bin_value[min_bin[1]] = bin_value[CV()] + Nvl (item_value[ITERATION_NUMBER + :N_BINS + 1], 0)
    )
)
 WHERE item_name IS NOT NULL
 ORDER BY item_value DESC

My Model solution works for any number of bins, passing the number of bins as a bind variable. The key idea here is to use values in the first N rows of a generic bin value measure to store all the running bin values, rather than as individual measures. I have included two modifications suggested by Stew in the AskTom thread.

  • Dimension by row number, ordered by item value
  • Initialise a bin measure to the row number (the first N items will remain fixed)
  • Initialise a bin value measure to item value (only first N entries used)
  • Add the row number as a measure, rn_m, in addition to a dimension, for referencing purposes
  • Add a min_bin measure for current minimum bin index (first entry only)
  • Add a measure for the number of iterations required, n_iters
  • The first N items are correctly binned in the measure initialisation
  • Set the minimum bin index using analytic Min function with KEEP clause over the first N rows of bin value
  • Set the bin for the current item to this index
  • Update the bin value for the corresponding bin only

Binfit, v1.3 - MOD

Recursive Subquery Factor for GBR

WITH bins AS (
       SELECT LEVEL bin, :N_BINS n_bins FROM DUAL CONNECT BY LEVEL <= :N_BINS
), items_desc AS (
       SELECT item_name, item_value, Row_Number () OVER (ORDER BY item_value DESC) rn
         FROM items
), rsf (bin, item_name, item_value, bin_value, lev, bin_rank, n_bins) AS (
SELECT b.bin,
       i.item_name, 
       i.item_value, 
       i.item_value,
       1,
       b.n_bins - i.rn + 1,
       b.n_bins
  FROM bins b
  JOIN items_desc i
    ON i.rn = b.bin
 UNION ALL
SELECT r.bin,
       i.item_name, 
       i.item_value, 
       r.bin_value + i.item_value,
       r.lev + 1,
       Row_Number () OVER (ORDER BY r.bin_value + i.item_value),
       r.n_bins
  FROM rsf r
  JOIN items_desc i
    ON i.rn = r.bin_rank + r.lev * r.n_bins
)
SELECT r.item_name,
       r.bin, r.item_value, r.bin_value
  FROM rsf r
 ORDER BY item_value DESC

The idea here is to use recursive subquery factors to iterate through the items in batches of N items, assigning each item to a bin according to the rank of the bin on the previous iteration.

  • Initial subquery factors form record sets for the bins and for the items with their ranks in descending order of value
  • The anchor branch assign bins to the first N items, assigning the item values to a bin value field, and setting the bin rank in ascending order of this bin value
  • The recursive branch joins the batch of items to the record in the previous batch whose bin rank matches that of the item in the reverse sense (so largest item goes to smallest bin etc.)
  • The analytic Row_Number function computes the updated bin ranks, and the bin values are updated by simple addition

Binfit, v1.3 - RSF

Recursive Subquery Factor for GBR with Temporary Table
Create Table and Index

DROP TABLE items_desc_temp
/
CREATE GLOBAL TEMPORARY TABLE items_desc_temp (
   item_name  VARCHAR2(30) NOT NULL,  
   item_value NUMBER(8) NOT NULL,
   rn         NUMBER
)
ON COMMIT DELETE ROWS
/
CREATE INDEX items_desc_temp_N1 ON items_desc_temp (rn)
/

Insert into Temporary Table

INSERT INTO items_desc_temp
SELECT item_name, item_value, Row_Number () OVER (ORDER BY item_value DESC) rn
  FROM items;

RSF Query with Temporary Table

WITH bins AS (
       SELECT LEVEL bin, :N_BINS n_bins FROM DUAL CONNECT BY LEVEL <= :N_BINS
), rsf (bin, item_name, item_value, bin_value, lev, bin_rank, n_bins) AS (
SELECT b.bin,
       i.item_name, 
       i.item_value, 
       i.item_value,
       1,
       b.n_bins - i.rn + 1,
       b.n_bins
  FROM bins b
  JOIN items_desc_temp i
    ON i.rn = b.bin
 UNION ALL
SELECT r.bin,
       i.item_name, 
       i.item_value, 
       r.bin_value + i.item_value,
       r.lev + 1,
       Row_Number () OVER (ORDER BY r.bin_value + i.item_value),
       r.n_bins
  FROM rsf r
  JOIN items_desc_temp i
    ON i.rn = r.bin_rank + r.lev * r.n_bins
)
SELECT item_name, bin, item_value, bin_value
  FROM rsf
 ORDER BY item_value DESC

The idea here is that in the initial RSF query a subquery factor of items was joined on a calculated field, so the whole record set had to be read, and performance could be improved by putting that initial record set into an indexed temporary table ahead of the main query. We'll see in the performance testing section that this changes quadratic variation with problem size into linear variation.

Plain Old SQL Solution for TPA

WITH items_desc AS (
       SELECT item_name, item_value, 
              Mod (Row_Number () OVER (ORDER BY item_value DESC), :N_BINS) + 1 bin
         FROM items
)
SELECT item_name, bin, item_value, Sum (item_value) OVER (PARTITION BY bin) bin_total
  FROM items_desc
 ORDER BY item_value DESC

The idea here is that the TPA algorithm can be implemented in simple SQL using analyic functions.

  • The subquery factor assigns the bins by taking the item rank in descending order of value and applying the modulo (N) function
  • The main query returns the bin totals in addition by analytic summing by bin

Pipelined Function for GDY
Package

CREATE OR REPLACE PACKAGE Bin_Fit AS

TYPE bin_fit_rec_type IS RECORD (item_name VARCHAR2(100), item_value NUMBER, bin NUMBER);
TYPE bin_fit_list_type IS VARRAY(1000) OF bin_fit_rec_type;

TYPE bin_fit_cur_rec_type IS RECORD (item_name VARCHAR2(100), item_value NUMBER);
TYPE bin_fit_cur_type IS REF CURSOR RETURN bin_fit_cur_rec_type;

FUNCTION Items_Binned (p_items_cur bin_fit_cur_type, p_n_bins PLS_INTEGER) RETURN bin_fit_list_type PIPELINED;

END Bin_Fit;
/
CREATE OR REPLACE PACKAGE BODY Bin_Fit AS

c_big_value                 CONSTANT NUMBER := 100000000;
TYPE bin_fit_cur_list_type  IS VARRAY(100) OF bin_fit_cur_rec_type;

FUNCTION Items_Binned (p_items_cur bin_fit_cur_type, p_n_bins PLS_INTEGER) RETURN bin_fit_list_type PIPELINED IS

  l_min_bin              PLS_INTEGER := 1;
  l_min_bin_val             NUMBER;
  l_bins                    SYS.ODCINumberList := SYS.ODCINumberList();
  l_bin_fit_cur_rec         bin_fit_cur_rec_type;
  l_bin_fit_rec             bin_fit_rec_type;
  l_bin_fit_cur_list        bin_fit_cur_list_type;

BEGIN

  l_bins.Extend (p_n_bins);
  FOR i IN 1..p_n_bins LOOP
    l_bins(i) := 0;
  END LOOP;

  LOOP

    FETCH p_items_cur BULK COLLECT INTO l_bin_fit_cur_list LIMIT 100;
    EXIT WHEN l_bin_fit_cur_list.COUNT = 0;

    FOR j IN 1..l_bin_fit_cur_list.COUNT LOOP

      l_bin_fit_rec.item_name := l_bin_fit_cur_list(j).item_name;
      l_bin_fit_rec.item_value := l_bin_fit_cur_list(j).item_value;
      l_bin_fit_rec.bin := l_min_bin;

      PIPE ROW (l_bin_fit_rec);
      l_bins(l_min_bin) := l_bins(l_min_bin) + l_bin_fit_cur_list(j).item_value;

      l_min_bin_val := c_big_value;
      FOR i IN 1..p_n_bins LOOP

        IF l_bins(i) < l_min_bin_val THEN
          l_min_bin := i;
          l_min_bin_val := l_bins(i);
        END IF;

      END LOOP;

    END LOOP;

  END LOOP;

END Items_Binned;

SQL Query

SELECT item_name, bin, item_value, Sum (item_value) OVER (PARTITION BY bin) bin_value
  FROM TABLE (Bin_Fit.Items_Binned (
                     CURSOR (SELECT item_name, item_value FROM items ORDER BY item_value DESC), 
                     :N_BINS))
 ORDER BY item_value DESC

The idea here is that procedural algorithms can often be implemented more efficiently in PL/SQL than in SQL.

  • The first parameter to the function is a strongly-typed reference cursor
  • The SQL call passes in a SELECT statement wrapped in the CURSOR keyword, so the function can be used for any set of records that returns name and numeric value pairs
  • The item records are fetched in batches of 100 using the LIMIT clause to improves efficiency

Performance Testing
I tested performance of the various queries using my own benchmarking framework across grids of data points, with two data sets to split the queries into two sets based on performance.

Query Modifications for Performance Testing

  • The RSF query with staging table was run within a pipelined function in order to easily include the insert in the timings
  • A system context was used to pass the bind variables as the framework runs the queries from PL/SQL, not from SQL*Plus
  • I found that calculating the bin values using analytic sums, as in the code above, affected performance, so I removed this for clarity of results, outputting only item name, value and bin

Test Data Sets
For a given depth parameter, d, random numbers were inserted within the range 0-d for d-1 records. The insert was:

 INSERT INTO items
  SELECT 'item-' || n, DBMS_Random.Value (0, p_point_deep) FROM  
  (SELECT LEVEL n FROM DUAL CONNECT BY LEVEL < p_point_deep);

The number of bins was passed as a width parameter, but note that the original, linked Model solution, MODO, hard-codes the number of bins to 3.

Test Results

Data Set 1 - Small
This was used for the following queries:

  • MODO - Original Model for GDY
  • MODB - Brendan's Generic Model for GDY
  • RSFQ - Recursive Subquery Factor for GBR
 Depth         W3         W3         W3
Run Type=MODO
 D1000       1.03       1.77       1.05
 D2000       3.98       6.46       5.38
 D4000      15.79       20.7      25.58
 D8000      63.18      88.75      92.27
D16000      364.2     347.74     351.99
Run Type=MODB
 Depth         W3         W6        W12
 D1000        .27        .42        .27
 D2000          1       1.58       1.59
 D4000       3.86        3.8       6.19
 D8000      23.26      24.57      17.19
D16000      82.29      92.04      96.02
Run Type=RSFQ
 D1000       3.24       3.17       1.53
 D2000       8.58       9.68       8.02
 D4000      25.65      24.07      23.17
 D8000      111.3     108.25      98.33
D16000     471.17     407.65     399.99

Slice W3
The results show:

  • Quadratic variation of CPU time with number of items
  • Little variation of CPU time with number of bins, although RSFQ seems to show some decline
  • RSFQ is slightly slower than MODO, while my version of Model, MODB is about 4 times faster than MODO

Data Set 2 - Large
This was used for the following queries:

  • RSFT - Recursive Subquery Factor for GBR with Temporary Table
  • POSS - Plain Old SQL Solution for TPA
  • PLFN - Pipelined Function for GDY

This table gives the CPU times in seconds across the data set:

  Depth       W100      W1000     W10000
Run Type=PLFN
 D20000        .31       1.92      19.25
 D40000        .65       3.87      55.78
 D80000       1.28       7.72      92.83
D160000       2.67      16.59     214.96
D320000       5.29      38.68      418.7
D640000      11.61      84.57      823.9
Run Type=POSS
 D20000        .09        .13        .13
 D40000        .18        .21        .18
 D80000        .27        .36         .6
D160000        .74       1.07        .83
D320000       1.36       1.58       1.58
D640000       3.13       3.97       4.04
Run Type=RSFT
 D20000        .78        .78        .84
 D40000       1.41       1.54        1.7
 D80000       3.02       3.39       4.88
D160000       6.11       9.56       8.42
D320000      13.05      18.93      20.84
D640000      41.62      40.98      41.09

Slice W100

Slice W10000
The results show:

  • Linear variation of CPU time with number of items
  • Little variation of CPU time with number of bins for POSS and RSFT, but roughly linear variation for PLFN
  • These linear methods are much faster than the earlier quadratic ones for larger numbers of items
  • Its approximate proportionality of time to number of bins means that, while PLFN is faster than RSFT for small number of bins, it becomes slower from around 50 bins for our problem
  • The proportionality to number of bins for PLFN presumably arises from the step to find the bin of minimum value
  • The lack of proportionality to number of bins for RSFT may appear surprising since it performs a sort of the bins iteratively: However, while the work for this sort is likely to be proportional to the number of bins, the number of iterations is inversely proportional and thus cancels out the variation

Solution Quality

The methods reported above implement three underlying algorithms, none of which guarantees an optimal solution. In order to get an idea of how the quality compares, I created new versions of the second set of queries using analytic functions to output the difference between minimum and maximum bin values, with percentage of the maximum also output. I ran these on the same grid, and report below the results for the four corners.

Method:			PLFN		RSFT		POSS
Point:	W100/D20000
Diff/%:			72/.004%	72/.004%	19,825/1%
Point:	W100/D640000
Diff/%:			60/.000003%	60/.000003%	633499/.03%
Point:	W10000/D20000
Diff/%:			189/.9%		180/.9%		19,995/67%
Point:	W10000/D640000
Diff/%:			695/.003%	695/.003%	639,933/3%

The results indicate that GDY (Greedy Algorithm) and GBR (Greedy Algorithm with Batched Rebalancing) generally give very similar quality results, while TPA (Team Picking Algorithm) tends to be quite a lot worse.

Extended Problem: Finding the Number of Bins Required

An important extension to the problem is when the bins have fixed capacity, and it is desired to find the minimum number of bins, then spread the items evenly between them. As mentioned at the start, I posted extensions to two of my solutions on an OTN thread, and I reproduce them here. It turns out to be quite easy to make the extension. The remainder of this section is just lifted from my OTN post and refers to the table of the original poster.

Start OTN Extract
So how do we determine the number of bins? The total quantity divided by bin capacity, rounded up, gives a lower bound on the number of bins needed. The actual number required may be larger, but mostly it will be within a very small range from the lower bound, I believe (I suspect it will nearly always be the lower bound). A good practical solution, therefore, would be to compute the solutions for a base number, plus one or more increments, and this can be done with negligible extra work (although Model might be an exception, I haven't tried it). Then the bin totals can be computed, and the first solution that meets the constraints can be used. I took two bin sets here.

SQL POS

WITH items AS (
       SELECT sl_pm_code item_name, sl_wt item_amt, sl_qty item_qty,
              Ceil (Sum(sl_qty) OVER () / :MAX_QTY) n_bins
         FROM ow_ship_det
), items_desc AS (
       SELECT item_name, item_amt, item_qty, n_bins,
              Mod (Row_Number () OVER (ORDER BY item_qty DESC), n_bins) bin_1,
              Mod (Row_Number () OVER (ORDER BY item_qty DESC), n_bins + 1) bin_2
         FROM items
)
SELECT item_name, item_amt, item_qty, 
       CASE bin_1 WHEN 0 THEN n_bins ELSE bin_1 END bin_1, 
       CASE bin_2 WHEN 0 THEN n_bins + 1 ELSE bin_2 END bin_2, 
       Sum (item_amt) OVER (PARTITION BY bin_1) bin_1_amt,
       Sum (item_qty) OVER (PARTITION BY bin_1) bin_1_qty,
       Sum (item_amt) OVER (PARTITION BY bin_2) bin_2_amt,
       Sum (item_qty) OVER (PARTITION BY bin_2) bin_2_qty
  FROM items_desc
 ORDER BY item_qty DESC, bin_1, bin_2

SQL Pipelined

SELECT osd.sl_pm_code item_name, osd.sl_wt item_amt, osd.sl_qty item_qty, 
       tab.bin_1, tab.bin_2, 
       Sum (osd.sl_wt) OVER (PARTITION BY tab.bin_1) bin_1_amt,
       Sum (osd.sl_qty) OVER (PARTITION BY tab.bin_1) bin_1_qty,
       Sum (osd.sl_wt) OVER (PARTITION BY tab.bin_2) bin_2_amt,
       Sum (osd.sl_qty) OVER (PARTITION BY tab.bin_2) bin_2_qty
  FROM ow_ship_det osd
  JOIN TABLE (Bin_Even.Items_Binned (
                     CURSOR (SELECT sl_pm_code item_name, sl_qty item_value,
                                    Sum(sl_qty) OVER () item_total
                               FROM ow_ship_det
                              ORDER BY sl_qty DESC, sl_wt DESC),
                     :MAX_QTY)) tab
    ON tab.item_name = osd.sl_pm_code
 ORDER BY osd.sl_qty DESC, tab.bin_1

Pipelined Function

CREATE OR REPLACE PACKAGE Bin_Even AS

TYPE bin_even_rec_type IS RECORD (item_name VARCHAR2(100), item_value NUMBER, bin_1 NUMBER, bin_2 NUMBER);
TYPE bin_even_list_type IS VARRAY(1000) OF bin_even_rec_type;

TYPE bin_even_cur_rec_type IS RECORD (item_name VARCHAR2(100), item_value NUMBER, item_total NUMBER);
TYPE bin_even_cur_type IS REF CURSOR RETURN bin_even_cur_rec_type;

FUNCTION Items_Binned (p_items_cur bin_even_cur_type, p_bin_max NUMBER) RETURN bin_even_list_type PIPELINED;

END Bin_Even;
/
SHO ERR
CREATE OR REPLACE PACKAGE BODY Bin_Even AS

c_big_value                 CONSTANT NUMBER := 100000000;
c_n_bin_sets                CONSTANT NUMBER := 2;

TYPE bin_even_cur_list_type IS VARRAY(100) OF bin_even_cur_rec_type;
TYPE num_lol_list_type      IS VARRAY(100) OF SYS.ODCINumberList;

FUNCTION Items_Binned (p_items_cur bin_even_cur_type, p_bin_max NUMBER) RETURN bin_even_list_type PIPELINED IS

  l_min_bin                 SYS.ODCINumberList := SYS.ODCINumberList (1, 1);
  l_min_bin_val             SYS.ODCINumberList := SYS.ODCINumberList (c_big_value, c_big_value);
  l_bins                    num_lol_list_type := num_lol_list_type (SYS.ODCINumberList(), SYS.ODCINumberList());

  l_bin_even_cur_rec        bin_even_cur_rec_type;
  l_bin_even_rec            bin_even_rec_type;
  l_bin_even_cur_list       bin_even_cur_list_type;

  l_n_bins                  PLS_INTEGER;
  l_n_bins_base             PLS_INTEGER;
  l_is_first_fetch          BOOLEAN := TRUE;

BEGIN

  LOOP

    FETCH p_items_cur BULK COLLECT INTO l_bin_even_cur_list LIMIT 100;
    EXIT WHEN l_Bin_Even_cur_list.COUNT = 0;
    IF l_is_first_fetch THEN

      l_n_bins_base := Ceil (l_Bin_Even_cur_list(1).item_total / p_bin_max) - 1;

      l_is_first_fetch := FALSE;

      l_n_bins := l_n_bins_base;
      FOR i IN 1..c_n_bin_sets LOOP

        l_n_bins := l_n_bins + 1;
        l_bins(i).Extend (l_n_bins);
        FOR k IN 1..l_n_bins LOOP
          l_bins(i)(k) := 0;
        END LOOP;

      END LOOP;

    END IF;

    FOR j IN 1..l_Bin_Even_cur_list.COUNT LOOP

      l_bin_even_rec.item_name := l_bin_even_cur_list(j).item_name;
      l_bin_even_rec.item_value := l_bin_even_cur_list(j).item_value;
      l_bin_even_rec.bin_1 := l_min_bin(1);
      l_bin_even_rec.bin_2 := l_min_bin(2);

      PIPE ROW (l_bin_even_rec);

      l_n_bins := l_n_bins_base;
      FOR i IN 1..c_n_bin_sets LOOP
        l_n_bins := l_n_bins + 1;
        l_bins(i)(l_min_bin(i)) := l_bins(i)(l_min_bin(i)) + l_Bin_Even_cur_list(j).item_value;

        l_min_bin_val(i) := c_big_value;
        FOR k IN 1..l_n_bins LOOP

          IF l_bins(i)(k) < l_min_bin_val(i) THEN
            l_min_bin(i) := k;
            l_min_bin_val(i) := l_bins(i)(k);
          END IF;

        END LOOP;

      END LOOP;

    END LOOP;

  END LOOP;

END Items_Binned;

END Bin_Even;

Output POS
Note BIN_1 means bin set 1, which turns out to have 4 bins, while bin set 2 then necessarily has 5.

ITEM_NAME         ITEM_AMT   ITEM_QTY      BIN_1      BIN_2  BIN_1_AMT  BIN_1_QTY  BIN_2_AMT  BIN_2_QTY
--------------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- ----------
1239606-1080          4024        266          1          1      25562        995      17482        827
1239606-1045          1880        192          2          2      19394        886      14568        732
1239606-1044          1567        160          3          3      18115        835      14097        688
1239606-1081          2118        140          4          4      18988        793      17130        657
1239606-2094          5741         96          1          5      25562        995      18782        605
...
1239606-2107            80          3          4          2      18988        793      14568        732
1239606-2084           122          3          4          3      18988        793      14097        688
1239606-2110           210          2          2          3      19394        886      14097        688
1239606-4022           212          2          3          4      18115        835      17130        657
1239606-4021           212          2          4          5      18988        793      18782        605

Output Pipelined

ITEM_NAME         ITEM_AMT   ITEM_QTY      BIN_1      BIN_2  BIN_1_AMT  BIN_1_QTY  BIN_2_AMT  BIN_2_QTY
--------------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- ----------
1239606-1080          4024        266          1          1      20627        878      15805        703
1239606-1045          1880        192          2          2      18220        877      16176        703
1239606-1044          1567        160          3          3      20425        878      15651        701
1239606-1081          2118        140          4          4      22787        876      14797        701
1239606-2094          5741         96          4          5      22787        876      19630        701
...
1239606-2089            80          3          4          1      22787        876      15805        703
1239606-2112           141          3          4          2      22787        876      16176        703
1239606-4022           212          2          1          1      20627        878      15805        703
1239606-4021           212          2          2          1      18220        877      15805        703
1239606-2110           210          2          3          2      20425        878      16176        703

End OTN Extract

Conclusions

  • Various solutions for the balanced number partitioning problem have been presented, using Oracle's Model clause, Recursive Subquery Factoring, Pipelined Functions and simple SQL
  • The performance characteristics of these solutions have been tested across a range of data sets
  • As is often the case, the best solution depends on the shape and size of the data set
  • A simple extension has been shown to allow determining the number of bins required in the bin-fitting interpretation of the problem
  • Replacing a WITH clause with a staging table can be a useful technique to allow indexed scans






Notes on Profiling Oracle PL/SQL

'Everything should be made as simple as possible, but not simpler'

This phrase is often attributed to Albert Einstein, although the attribution is apparently questionable:
Everything Should Be Made as Simple as Possible, But Not Simpler. In any case it's not a bad approach to follow, even if the quote did come from a non-Oracle guy :).

I recently started looking at the hierarchical profiler tool with a view to using it in an upcoming project. In order to understand the tool properly, I felt it would be a good idea to start by using it to profile a test program that would be as simple as possible while covering as wide a range of scenarios as possible. This article documents the results of that profiling, highlighting the different scenarios covered, discusses the output from the profiler, and includes a query I wrote to display the function call tree.

The article goes on to illustrate profiling through manual code instrumentation, and by the old flat profiler (DBMS_Profiler) on the same test program, concluding that each method has its own strengths and weaknesses.

Setup
The hierarchical profiler setup and use is described in Oracle® Database Advanced Application Developer's Guide 11g Release 2 (11.2), and some code snippets are available here:PL/SQL Hierarchical Profiler in Oracle Database 11g Release 1

Scenarios
The test program consists of a driving script, Test_Rep_p.sql (attached), that calls a package (HProf_Test) and an object type (Table_Count_Type), both defined in the attached script, HProf_Test_Code.sql. The test program covers the following scenarios:

  • Multiple root calls (__plsql_vm, A_CALLS_B)
  • Recursive procedure calls (procedure calling itself: R_CALLS_R)
  • Mutually recursive procedure calls (procedures call each other: A_CALLS_B and B_CALLS_A)
  • Procedure called by multiple procedures (child with multiple parents: PUT_LINE)
  • Procedure 'inlined' within PL/SQL (Rest_a_While)
  • Static SQL within PL/SQL (__static_sql_exec_line8)
  • Dynamic SQL within PL/SQL (__dyn_sql_exec_line12)
  • 'Everything should be made as simple as possible, but not simpler'

    This phrase is often attributed to Albert Einstein, although the attribution is apparently questionable:
    Everything Should Be Made as Simple as Possible, But Not Simpler. In any case it's not a bad approach to follow, even if the quote did come from a non-Oracle guy :).

    I recently started looking at the hierarchical profiler tool with a view to using it in an upcoming project. In order to understand the tool properly, I felt it would be a good idea to start by using it to profile a test program that would be as simple as possible while covering as wide a range of scenarios as possible. This article documents the results of that profiling, highlighting the different scenarios covered, discusses the output from the profiler, and includes a query I wrote to display the function call tree.

    The article goes on to illustrate profiling through manual code instrumentation, and by the old flat profiler (DBMS_Profiler) on the same test program, concluding that each method has its own strengths and weaknesses.

    Setup
    The hierarchical profiler setup and use is described in Oracle® Database Advanced Application Developer's Guide 11g Release 2 (11.2), and some code snippets are available here:PL/SQL Hierarchical Profiler in Oracle Database 11g Release 1

    Scenarios
    The test program consists of a driving script, Test_Rep_p.sql (attached), that calls a package (HProf_Test) and an object type (Table_Count_Type), both defined in the attached script, HProf_Test_Code.sql. The test program covers the following scenarios:

    • Multiple root calls (__plsql_vm, A_CALLS_B)
    • Recursive procedure calls (procedure calling itself: R_CALLS_R)
    • Mutually recursive procedure calls (procedures call each other: A_CALLS_B and B_CALLS_A)
    • Procedure called by multiple procedures (child with multiple parents: PUT_LINE)
    • Procedure 'inlined' within PL/SQL (Rest_a_While)
    • Static SQL within PL/SQL (__static_sql_exec_line8)
    • Dynamic SQL within PL/SQL (__dyn_sql_exec_line12)
    • Database function called from SQL in SQL*Plus (DBFUNC)
    • Database function called from SQL in PL/SQL (DBFUNC)
    • Object constructor call (TABLE_COUNT_TYPE)

    Call Structure Diagram
    HProf - CSD

    Raw Results
    The attached script Test_Rep_h.sql was used to report on the results. The record produced in the run table, DBMSHP_RUNS, was:

         RUNID RUN_TIMESTAMP                   MICRO_S    SECONDS RUN_COMMENT
    ---------- ---------------------------- ---------- ---------- ------------------------------------------------------------
            11 04-MAR-13 07.07.36.803000        890719        .89 Profile for small test program with recursion

    The records produced in the functions table, DBMSHP_FUNCTION_INFO, were:

    OWNER MODULE               FUNCTION                         ID  LINE#      SUB_T      FUN_T  CALLS
    ----- -------------------- ------------------------------ ---- ------ ---------- ---------- ------
    NET   HPROF_TEST           A_CALLS_B                         4     40      62340       4450      1
    NET   HPROF_TEST           A_CALLS_B@1                       5     40      43729      13663      1
    NET   HPROF_TEST           B_CALLS_A                         6     38      57890      14161      1
    NET   HPROF_TEST           B_CALLS_A@1                       7     38      30066      30066      1
    NET   HPROF_TEST           DBFUNC                            8     84      32629      32629      2
    NET   HPROF_TEST           R_CALLS_R                         9     70      12823       4159      1
    NET   HPROF_TEST           R_CALLS_R@1                      10     70       8633       8618      1
    NET   HPROF_TEST           STOP_PROFILING                   11     16         21         21      1
    NET   TABLE_COUNT_TYPE     TABLE_COUNT_TYPE                 12      3      55049         82      1
    NET   TABLE_COUNT_TYPE     __static_sql_exec_line6          22      6      54967      54967      1
    SYS   DBMS_HPROF           STOP_PROFILING                   13     59          0          0      1
    SYS   DBMS_OUTPUT          GET_LINE                         14    129          8          8      3
    SYS   DBMS_OUTPUT          GET_LINES                        15    160         68         60      3
    SYS   DBMS_OUTPUT          NEW_LINE                         16    117          7          7      2
    SYS   DBMS_OUTPUT          PUT                              17     77         28         28      2
    SYS   DBMS_OUTPUT          PUT_LINE                         18    109         46         11      2
                               __anonymous_block                 1      0     809839        521      5
                               __dyn_sql_exec_line12            19     12        226        226      1
                               __plsql_vm                        2      0     828379         58      6
                               __plsql_vm@1                      3      0      14158         11      1
                               __sql_fetch_line13               20     13     726713     726713      1
                               __static_sql_exec_line8          21      8      14418        260      1
    
    22 rows selected.

    The SUB_T and FUN_T values are the total times in microseconds for the subtree including function, and function-only processing.

    The records produced in the functions parent-child table, DBMSHP_PARENT_CHILD_INFO, were:

    OWNER_P MODULE_P             FUNCTION_P                     OWNER_C MODULE_C             FUNCTION_C                          SUB_T      FUN_T  CALLS
    ------- -------------------- ------------------------------ ------- -------------------- ------------------------------ ---------- ---------- ------
    NET     HPROF_TEST           STOP_PROFILING                 SYS     DBMS_HPROF           STOP_PROFILING                          0          0      1
    NET     HPROF_TEST           R_CALLS_R@1                    SYS     DBMS_OUTPUT          PUT_LINE                               15          6      1
    NET     HPROF_TEST           R_CALLS_R                      SYS     DBMS_OUTPUT          PUT_LINE                               31          5      1
    NET     HPROF_TEST           R_CALLS_R                      NET     HPROF_TEST           R_CALLS_R@1                          8633       8618      1
    NET     HPROF_TEST           B_CALLS_A                      NET     HPROF_TEST           A_CALLS_B@1                         43729      13663      1
    NET     HPROF_TEST           A_CALLS_B@1                    NET     HPROF_TEST           B_CALLS_A@1                         30066      30066      1
    NET     HPROF_TEST           A_CALLS_B                      NET     HPROF_TEST           B_CALLS_A                           57890      14161      1
    NET     TABLE_COUNT_TYPE     TABLE_COUNT_TYPE               NET     TABLE_COUNT_TYPE     __static_sql_exec_line6             54967      54967      1
    SYS     DBMS_OUTPUT          PUT_LINE                       SYS     DBMS_OUTPUT          NEW_LINE                                7          7      2
    SYS     DBMS_OUTPUT          PUT_LINE                       SYS     DBMS_OUTPUT          PUT                                    28         28      2
    SYS     DBMS_OUTPUT          GET_LINES                      SYS     DBMS_OUTPUT          GET_LINE                                8          8      3
                                 __anonymous_block              NET     HPROF_TEST           STOP_PROFILING                         21         21      1
                                 __anonymous_block              SYS     DBMS_OUTPUT          GET_LINES                              68         60      3
                                 __anonymous_block                                           __dyn_sql_exec_line12                 226        226      1
                                 __anonymous_block              NET     HPROF_TEST           R_CALLS_R                           12823       4159      1
                                 __anonymous_block                                           __static_sql_exec_line8             14418        260      1
                                 __plsql_vm                     NET     HPROF_TEST           DBFUNC                              18482      18482      1
                                 __anonymous_block                                           __sql_fetch_line13                 726713     726713      1
                                 __static_sql_exec_line8                                     __plsql_vm@1                        14158         11      1
                                 __plsql_vm                                                  __anonymous_block                  809839        521      5
                                 __plsql_vm@1                   NET     HPROF_TEST           DBFUNC                              14147      14147      1
                                 __anonymous_block              NET     TABLE_COUNT_TYPE     TABLE_COUNT_TYPE                    55049         82      1
    
    22 rows selected.

    The SUB_T and FUN_T values are the total times in microseconds for the subtree including function, and function-only processing, respectively, for the child function while called from all instances of the parent.

    Function Call Tree
    The raw data above can be used to identify processing bottlenecks at a function level, but it's also useful to process the data in order to display the function hierarchies, both for performance tuning and also for understanding the program structure. This is not quite as trivial as it may seem. The oracle-base article provides an SQL statement that attempts to do this:

    SELECT RPAD(' ', level*2, ' ') || fi.owner || '.' || fi.module AS name,
           fi.function,
           pci.subtree_elapsed_time,
           pci.function_elapsed_time,
           pci.calls
    FROM   dbmshp_parent_child_info pci
           JOIN dbmshp_function_info fi ON pci.runid = fi.runid AND pci.childsymid = fi.symbolid
    WHERE  pci.runid = :RUN_ID
    CONNECT BY PRIOR childsymid = parentsymid
    START WITH pci.parentsymid = :START_ID

    Here, bind variables replace the original hard-coded values. On running this query I often got the following result:

    ERROR at line 1:
    ORA-01436: CONNECT BY loop in user data

    On the run used in this article, the query returned 157 records, which is obviously incorrect. There is of course a NOCYCLE keyword that can be used to return results in the case of loops. However, it is not worth adding in this case, because there are in fact no loops in the data (at least no cyclic loops - apparent loops are discussed later). Oracle avoids loops by treating a function call that is a descendant of itself as a call to a new function, identified by suffices @1, @2 etc. as we can see from the recursive procedures above (eg R_CALLS_R@1 is the second call of R_CALLS_R, this one from itself). The problem here is that the query is incorrect in its handling of runid, with the result that the tree-walk traverses records from other runs as well as the intended one. A further problem is that there may be several roots, and it would be best to calculate these within a subquery. We can correct these problems by the following query:

    SELECT RPAD(' ', level*2, ' ') || fi.owner || '.' || fi.module AS name,
           fi.symbolid || ': ' || fi.function function,
           pci.subtree_elapsed_time sub_t,
           pci.function_elapsed_time fun_t,
           pci.calls
      FROM dbmshp_parent_child_info	pci
      JOIN dbmshp_function_info		fi 
        ON pci.runid	              = fi.runid 
       AND pci.childsymid	       = fi.symbolid
     WHERE pci.runid                   = :RUN_ID
    CONNECT BY PRIOR pci.childsymid    = pci.parentsymid 
           AND pci.runid	              = :RUN_ID
    START WITH pci.parentsymid         IN (SELECT f.symbolid FROM dbmshp_function_info f WHERE NOT EXISTS 
           (SELECT 1 FROM dbmshp_parent_child_info i WHERE i.childsymid = f.symbolid AND i.runid = :RUN_ID) AND f.runid = :RUN_ID)
           AND pci.runid	              = :RUN_ID

    This query returns the results:

    NAME                           FUNCTION                            SUB_T      FUN_T  CALLS
    ------------------------------ ------------------------------ ---------- ---------- ------
      .                            1: __anonymous_block              809,839        521      5
        NET.HPROF_TEST             9: R_CALLS_R                       12,823      4,159      1
          NET.HPROF_TEST           10: R_CALLS_R@1                     8,633      8,618      1
            SYS.DBMS_OUTPUT        18: PUT_LINE                           15          6      1
              SYS.DBMS_OUTPUT      16: NEW_LINE                            7          7      2
              SYS.DBMS_OUTPUT      17: PUT                                28         28      2
          SYS.DBMS_OUTPUT          18: PUT_LINE                           31          5      1
            SYS.DBMS_OUTPUT        16: NEW_LINE                            7          7      2
            SYS.DBMS_OUTPUT        17: PUT                                28         28      2
        NET.HPROF_TEST             11: STOP_PROFILING                     21         21      1
          SYS.DBMS_HPROF           13: STOP_PROFILING                      0          0      1
        NET.TABLE_COUNT_TYPE       12: TABLE_COUNT_TYPE               55,049         82      1
          NET.TABLE_COUNT_TYPE     22: __static_sql_exec_line6        54,967     54,967      1
        SYS.DBMS_OUTPUT            15: GET_LINES                          68         60      3
          SYS.DBMS_OUTPUT          14: GET_LINE                            8          8      3
        .                          19: __dyn_sql_exec_line12             226        226      1
        .                          20: __sql_fetch_line13            726,713    726,713      1
        .                          21: __static_sql_exec_line8        14,418        260      1
          .                        3: __plsql_vm@1                    14,158         11      1
            NET.HPROF_TEST         8: DBFUNC                          14,147     14,147      1
      NET.HPROF_TEST               8: DBFUNC                          18,482     18,482      1
      NET.HPROF_TEST               6: B_CALLS_A                       57,890     14,161      1
        NET.HPROF_TEST             5: A_CALLS_B@1                     43,729     13,663      1
          NET.HPROF_TEST           7: B_CALLS_A@1                     30,066     30,066      1
    
    24 rows selected.

    This is better, but we can identify some further issues.

    Missing Roots
    The true root results are missing: For example, A_CALLS_B is missing. This arises because the query is traversing the link records (DBMSHP_PARENT_CHILD_INFO), while the root information is stored in the nodes (DBMSHP_FUNCTION_INFO). This suggests a change from the CONNECT BY syntax to Oracle's v11.2 recursive subquery factoring syntax, which allows you easily to start from the nodes, then traverse recursively via the links. (Incidentally, moving the start of profiling to its own block would result in A_CALLS_B appearing under __anonymous_block, but I prefer to retain the current structure in order to deal with the general case in which multiple roots are possible.)

    Duplicate Links
    Notice that function PUT_LINE is reported separately under R_CALLS_R and R_CALLS_R@1, and the timings differ. Also, its own child calls appear under each of its instances, but in those cases the timings are identical. The reason for this is that in the first case, there are separate records of the times used in each call, whereas in the second, the child calls have only a single record giving the total times across both instances of the parent call. The call from R_CALLS_R shows (9 - 4 = ) 5µs used in child calls, while the call from R_CALLS_R@1 shows 14µs. The child calls show totals of (3 + 16 = ) 19µs, equalling the sum across the parent calls.

    At this point it is worth looking at this from the more general perspective of a hierarchical data structure where parents can have multiple children and children multiple parents, with one or more roots. If a network diagram were constructed there would be loops apparent indicating multiple routes between nodes. In these situations, Oracle's hierarchical queries effectively traverse all routes, and this is what causes the link duplication (in other scenarios this behaviour can cause big performance problems, but probably not here). Oracle's cycle detection mechanism does not trigger because the loops do not result in any node being a descendant of itself (as noted above, extra nodes are generated by the profiler to avoid this).

    It seems to me better to avoid this duplication, and also to signal those cases where times are not aggregated up the tree. We can achieve this by the use of analytic functions. Note that, although the query below refers to the specific tables and attributes for this problem, the proposed solution could be used for any member of this general class of problem. The new query, which orders sibling records by descending subtree elapsed time, is:

    WITH last_run AS (
    SELECT Max (runid) runid FROM dbmshp_runs
    ), full_tree (runid, lev, node_id, sub_t, fun_t, calls, link_id) AS (
    SELECT fni.runid, 0, fni.symbolid, fni.subtree_elapsed_time, fni.function_elapsed_time, fni.calls, 'root' || ROWNUM
      FROM dbmshp_function_info fni
      JOIN last_run lrn
        ON lrn.runid = fni.runid
     WHERE NOT EXISTS (SELECT 1 FROM dbmshp_parent_child_info pci WHERE pci.childsymid = fni.symbolid AND pci.runid = fni.runid)
     UNION ALL
    SELECT ftr.runid, 
           ftr.lev + 1, 
           pci.childsymid, 
           pci.subtree_elapsed_time, 
           pci.function_elapsed_time, 
           pci.calls,
           pci.parentsymid || '-' || pci.childsymid
      FROM full_tree ftr
      JOIN dbmshp_parent_child_info pci
        ON pci.parentsymid = ftr.node_id
       AND pci.runid = ftr.runid
    ) SEARCH DEPTH FIRST BY sub_t DESC, fun_t DESC, calls DESC, node_id SET rn
    , tree_ranked AS (
    SELECT runid, node_id, lev, rn, 
           sub_t, fun_t, calls, 
           Row_Number () OVER (PARTITION BY node_id ORDER BY rn) node_rn,
           Count (*) OVER (PARTITION BY node_id) node_cnt,
           Row_Number () OVER (PARTITION BY link_id ORDER BY rn) link_rn
      FROM full_tree
    )
    SELECT RPad (' ', trr.lev*2, ' ') || fni.function "Function tree",
           fni.symbolid sy, fni.owner, fni.module,
           CASE WHEN trr.node_cnt > 1 THEN trr.node_rn || ' of ' || trr.node_cnt END "Inst.",
           trr.sub_t, trr.fun_t, trr.calls, 
           trr.rn "Row"
      FROM tree_ranked trr
      JOIN dbmshp_function_info fni
        ON fni.symbolid = trr.node_id
       AND fni.runid = trr.runid
     WHERE trr.link_rn = 1
     ORDER BY trr.rn

    Query Structure Diagram
    HProf - QSD

    The results are then:

    Function tree                        SY OWNER MODULE               Inst.         SUB_T      FUN_T  CALLS  Row
    ----------------------------------- --- ----- -------------------- -------- ---------- ---------- ------ ----
    __plsql_vm                            2                                        828,379         58      6    1
      __anonymous_block                   1                                        809,839        521      5    2
        __sql_fetch_line13               20                                        726,713    726,713      1    3
        TABLE_COUNT_TYPE                 12 NET   TABLE_COUNT_TYPE                  55,049         82      1    4
          __static_sql_exec_line6        22 NET   TABLE_COUNT_TYPE                  54,967     54,967      1    5
        __static_sql_exec_line8          21                                         14,418        260      1    6
          __plsql_vm@1                    3                                         14,158         11      1    7
            DBFUNC                        8 NET   HPROF_TEST           1 of 2       14,147     14,147      1    8
        R_CALLS_R                         9 NET   HPROF_TEST                        12,823      4,159      1    9
          R_CALLS_R@1                    10 NET   HPROF_TEST                         8,633      8,618      1   10
            PUT_LINE                     18 SYS   DBMS_OUTPUT          1 of 2           15          6      1   11
              PUT                        17 SYS   DBMS_OUTPUT          1 of 2           28         28      2   12
              NEW_LINE                   16 SYS   DBMS_OUTPUT          1 of 2            7          7      2   13
          PUT_LINE                       18 SYS   DBMS_OUTPUT          2 of 2           31          5      1   14
        __dyn_sql_exec_line12            19                                            226        226      1   17
        GET_LINES                        15 SYS   DBMS_OUTPUT                           68         60      3   18
          GET_LINE                       14 SYS   DBMS_OUTPUT                            8          8      3   19
        STOP_PROFILING                   11 NET   HPROF_TEST                            21         21      1   20
          STOP_PROFILING                 13 SYS   DBMS_HPROF                             0          0      1   21
      DBFUNC                              8 NET   HPROF_TEST           2 of 2       18,482     18,482      1   22
    A_CALLS_B                             4 NET   HPROF_TEST                        62,340      4,450      1   23
      B_CALLS_A                           6 NET   HPROF_TEST                        57,890     14,161      1   24
        A_CALLS_B@1                       5 NET   HPROF_TEST                        43,729     13,663      1   25
          B_CALLS_A@1                     7 NET   HPROF_TEST                        30,066     30,066      1   26
    
    24 rows selected.

    Notice that we now have a single record for each of the 22 links, plus the two root nodes. Also, the "Inst." column lists the instance number of a function having more than one instance, and the children of any such function are only listed once with the gaps in the "Row" column indicating where duplicates have been suppressed.

    Network Diagrams
    It may be interesting to display the call tree in two diagrams, one for each root.
    Root __plsql_vm
    HProf - Net

    Root A_CALLS_B
    HProf - Net2

    Notes on Tree Output
    Anonymous Block (__anonymous_block)
    This function seems to correspond to invocations of anonymous blocks, obviously enough. However, there is an apparent anomaly in the number of calls listed, 6, because the driving program has only three such blocks, and there are none in the called PL/SQL code. I would surmise that the apparent discrepancy arises from the enabling of SERVEROUTPUT, which appears to result in a secondary block being associated with each explicit SQL*Plus block, that issues a call to GET_LINES to process buffered output.

    PL/SQL Engine (__plsql_vm)
    This function seems to correspond to external invocations of PL/SQL such as from a SQL*Plus session. There are 7 calls, 6 of them presumably being linked with the external anonymous blocks, and the seventh with DBFUNC, where a PL/SQL function is called from a SQL statement from SQL*Plus.

    Notice that the SQL statement calling a database function from within PL/SQL generates the recursive call to the engine, __plsql_vm@1

    Second Root (A_CALLS_B)
    The above function does not have the __plsql_vm/__anonymous_block ancestry that might be expected because profiling only started within the enclosing block.

    Inlined Procedure (Rest_a_While)
    I wrote a small procedure, Rest_a_While, to generate some elapsed time in the recursive procedures, but preceded it with the INLINE pragma, a new optimisation feature in 11g. This had the desired effect of removing the calls from the profiling output and including the times in the calling procedures. Rest_a_While does not make the obvious call to DBMS_Lock.Sleep because that procedure cannot be inlined. subprogram inlining in 11g provides some analysis of the inlining feature.

    Sibling Ordering
    We have ordered siblings by descending subtree elapsed time, using the SEARCH clause. It would be nice to have the option to order the siblings by initial invocation time, but Oracle does not provide the data to do this.

    Loops and Hierarchies
    The first diagram shows two loops, where there are two routes between the loop start and end points, indicated by different colours. The second loop has two child nodes coming from the end point, and hierarchical queries (both CONNECT BY and recursive subquery factors in Oracle) cause the links to be duplicated. Our query has filtered out the duplicates by analytic functions.

    It's worth remembering this because it's a general feature of SQL for querying hierarchies, and judging by Oracle forums, not one that's widely understood. For larger hierarchies it can cause serious performance problems, and may justify a PL/SQL programmed solution that need not suffer the same problem.

    Manual Instrumentation
    Oracle's hierarchical profiler clearly provides extremely useful information on both performance and structure of PL/SQL programs with very little effort. However, it does have the limitation of only providing information down to the subprogram level (which includes embedded SQL statements in this context). It is also often considered good practice to implement timing and other instrumentation permanently in production code, sometimes in a switchable fashion. In the test program, one of the called procedures, A_Calls_B, makes two calls to the inlined procedure, Rest_a_While, the second doing about twice as much work as the first. The profiler reports total within-function times of 4,450µs and 13,663µs on first and second calls, respectively (the work is scaled by a call number parameter, equal to 1, then 3).

    I created a second instance of the package and driver script (suffix _TS) to illustrate manual instrumentation. This uses an 'object-oriented' timing package that I wrote a couple of years ago Code Timing and Object Orientation and Zombies (November, 2010) to instrument at procedure and section level. I multiplied the work in Rest_a_While by a factor of ten to get larger times. This produced the output:

    Timer Set: HProf, Constructed at 05 Mar 2013 10:21:27, written at 10:21:30
    ==========================================================================
    [Timer timed: Elapsed (per call): 0.04 (0.000044), CPU (per call): 0.05 (0.000050), calls: 1000, '***' denotes corrected line below]
    
    Timer                       Elapsed          CPU          Calls        Ela/Call        CPU/Call
    ----------------------   ----------   ----------   ------------   -------------   -------------
    A_Calls_B, section one         0.06         0.05              2         0.03150         0.02500
    A_Calls_B, section two         0.12         0.12              2         0.06050         0.06000
    B_Calls_A: 2                   0.15         0.16              1         0.15400         0.16000
    B_Calls_A: 4                   0.31         0.30              1         0.30700         0.30000
    DBFunc                         0.32         0.31              2         0.15950         0.15500
    Open cursor                    0.69         0.69              1         0.68900         0.69000
    Fetch from cursor              0.70         0.70              1         0.69600         0.70000
    Close cursor                   0.00         0.00              1         0.00000         0.00000
    Construct object               0.06         0.04              1         0.05500         0.04000
    R_Calls_R                      0.14         0.14              2         0.07000         0.07000
    (Other)                        0.00         0.00              1         0.00000         0.00000
    ----------------------   ----------   ----------   ------------   -------------   -------------
    Total                          2.54         2.51             15         0.16960         0.16733
    ----------------------   ----------   ----------   ------------   -------------   -------------
    

    Notes on Code Timing

    • Calls, CPU and elapsed times have been captured at the section level for A_Calls_B
    • Observe that, while R_Calls_R and A_Calls_B aggregate over all calls, B_Calls_A records values by call; this is implemented simply by including a value that changes with call in the timer name
    • The timing set object is designed to be very low footprint; here 9 statements (calls to Increment_Time), plus a small global overhead, produced 10 result lines, plus associated information
    • The 'object-oriented' approach allows multiple programs to be be timed at multiple levels, without interference between timings
    • There are Perl and Java implementations of this timing set object included in the Scribd article mentioned

    Oracle's Flat Profiler (DBMS_Profiler)
    The hierarchical profiler was introduced in v11.1, while prior to this there was a non-hierarchical profiler, DBMS_Profiler. This package still exists in v11: It is omitted from the advanced application developer's guide for v11, but is described in the packages and types manual (Oracle® Database PL/SQL Packages and Types Reference, 11g Release 2 (11.2)); also, SQL*Developer appears to support only the newer hierarchical verion (via right-click on a package). I thought it interesting to run the older version on the same test program (package Old_Test_Prof, driver script Test_Rep_p_Old.sql and reporting script Test_Rep_h_Old.sql). The output from the first three queries is:

    Run header (PLSQL_PROFILER_RUNS)
    
         RUNID RUN_DATE        MICRO_S    SECONDS
    ---------- ------------ ---------- ----------
             3 11:03:13        2164000       2.16
    
    Profiler data summary (PLSQL_PROFILER_DATA)
    
       MICRO_S SECONDS    CALLS
    ---------- ------- --------
       2126949    2.13       72
    
    Profiler data by time (PLSQL_PROFILER_DATA)
    
       MICRO_S SECONDS    CALLS UNIT_NAME            UNIT_NUMBER  LINE#
    ---------- ------- -------- -------------------- ----------- ------
        729932    0.73        1                                5     13
        569563    0.57        2 OLD_PROF_TEST                  1     56
        377880    0.38        2 OLD_PROF_TEST                  1     82
        166019    0.17        2 OLD_PROF_TEST                  1     70
        150117    0.15        2 OLD_PROF_TEST                  1     43
         72742    0.07        2 OLD_PROF_TEST                  1     40
         56473    0.06        1 TABLE_COUNT_TYPE               6      6
          3338    0.00        1                                5      8
           258    0.00        1                                5     12
           109    0.00        1                                5     16
            68    0.00        2 OLD_PROF_TEST                  1     67
            66    0.00        2                                4      1
            60    0.00        2                                7      1
            60    0.00        2                                3      1
            44    0.00        1                                5     14
            42    0.00        0                                2      5
            31    0.00        1 OLD_PROF_TEST                  1     18
            26    0.00        1                                8      5
            13    0.00        0                                5      1
             9    0.00        1 TABLE_COUNT_TYPE               6     11
             9    0.00        2 OLD_PROF_TEST                  1     86
             8    0.00        0 OLD_PROF_TEST                  1     51
             8    0.00        1 TABLE_COUNT_TYPE               6     13
             7    0.00        1                                5     18
             6    0.00        0 OLD_PROF_TEST                  1     78
             6    0.00        0 OLD_PROF_TEST                  1     64
             6    0.00        0                                8      1
             6    0.00        1 TABLE_COUNT_TYPE               6      3
             5    0.00        0 OLD_PROF_TEST                  1     35
             5    0.00        0 OLD_PROF_TEST                  1     15
             4    0.00        1                                8      7
             4    0.00        1 OLD_PROF_TEST                  1     76
             3    0.00        1                                2      8
             2    0.00        1 OLD_PROF_TEST                  1     62
             2    0.00        1 OLD_PROF_TEST                  1     13
             2    0.00        1 TABLE_COUNT_TYPE               6      5
             2    0.00        2 OLD_PROF_TEST                  1     72
             2    0.00        2 OLD_PROF_TEST                  1     45
             2    0.00        2 OLD_PROF_TEST                  1     49
             2    0.00        2 OLD_PROF_TEST                  1     46
             2    0.00        2 OLD_PROF_TEST                  1     58
             1    0.00        1 OLD_PROF_TEST                  1     73
             1    0.00        1                                2      6
             1    0.00        1 OLD_PROF_TEST                  1     59
             1    0.00        1 OLD_PROF_TEST                  1     11
             1    0.00        2 OLD_PROF_TEST                  1     54
             1    0.00        2 OLD_PROF_TEST                  1     84
             0    0.00        0 OLD_PROF_TEST                  1      1
             0    0.00        0 OLD_PROF_TEST                  1     88
             0    0.00        0                                8      9
             0    0.00        0                                2      1
             0    0.00        0                                2      2
             0    0.00        0 OLD_PROF_TEST                  1      3
             0    0.00        0 OLD_PROF_TEST                  1      5
             0    0.00        0 OLD_PROF_TEST                  1      9
             0    0.00        0 OLD_PROF_TEST                  1     20
             0    0.00        1 TABLE_COUNT_TYPE               6      4
             0    0.00        1                                8      2
             0    0.00        2 OLD_PROF_TEST                  1     39
             0    0.00        2 OLD_PROF_TEST                  1     55
             0    0.00        2 OLD_PROF_TEST                  1     69
             0    0.00        2 OLD_PROF_TEST                  1     38
             0    0.00        2 OLD_PROF_TEST                  1     81
             0    0.00        2 OLD_PROF_TEST                  1     42
             0    0.00        2 OLD_PROF_TEST                  1     68
    
    65 rows selected.
    
    

    Referring to the package, type and anonymous blocks, I assigned labels to all the lines having more than 10µs, as follows:

       MICRO_S SECONDS    CALLS UNIT_NAME            UNIT_NUMBER  LINE#
    ---------- ------- -------- -------------------- ----------- ------
        729932    0.73        1                                5     13  B2: FETCH
        569563    0.57        2 OLD_PROF_TEST                  1     56  B_Calls_A (Rest_a_While)
        377880    0.38        2 OLD_PROF_TEST                  1     82  DBFunc (Rest_a_While)
        166019    0.17        2 OLD_PROF_TEST                  1     70  R_Calls_R (Rest_a_While)
        150117    0.15        2 OLD_PROF_TEST                  1     43  A_Calls_B (Rest_a_While, section 2)
         72742    0.07        2 OLD_PROF_TEST                  1     40  A_Calls_B (Rest_a_While, section 1)
         56473    0.06        1 TABLE_COUNT_TYPE               6      6  SELECT
          3338    0.00        1                                5      8  B2: SELECT DBFunc
           258    0.00        1                                5     12  B2: OPEN
           109    0.00        1                                5     16  B2: Assign Table_Count_Type
            68    0.00        2 OLD_PROF_TEST                  1     67  Put_Line
            66    0.00        2                                4      1  Auxiliary SERVEROUTPUT block for B2 (surmised)
            60    0.00        2                                7      1  Auxiliary SERVEROUTPUT block for B3 (surmised)
            60    0.00        2                                3      1  Auxiliary SERVEROUTPUT block for B1 (surmised)
            44    0.00        1                                5     14  B2: CLOSE
            42    0.00        0                                2      5  B1: Call to Start_Profiling 
            31    0.00        1 OLD_PROF_TEST                  1     18  RETURN DBMS_Profiler.Stop_Profiler;
            26    0.00        1                                8      5  B3: Call R_Calls_R
            13    0.00        0                                5      1  B2: DECLARE
    

    Notes on Output of Flat Profiler
    There were six units with no linked information in DBMS_PROFILER_UNITS. By examining the data, I was able to associate unit numbers 2, 5 and 8 with my anonymous blocks B1, B2 and B3. That left three unassigned, and I have surmised that these correspond to the auxiliary blocks associated with processing server output that we earlier surmised when examining the output from the hierarchical profiler.

    • The useful call tree structure is not present in the data from the old profiler
    • However, the results are at a line level, which the hierarchical profiler does not provide; for example, the two sections of A_Calls_B are reported separately
    • Deciphering the output requires significantly more manual effort than with the hierarchical profiler
    • Both old and new profiler have their own advantages, and so both should be considered of value
    • Manual code timing offers more flexibility in terms of aggregating lines and call instances, but requires more effort

    Conclusions

    • Running Oracle's hierarchical profiler would seem to be the default first step in tuning PL/SQL programs from v11.1
    • Some care is needed in interpreting the output data; I've provided a query for displaying the hierarchies
    • Performance is recorded only down to function level, so it will still often be worthwhile to use the old flat profiler in addition
    • Manually timing code sections also still has a part to play, in terms of instrumentation and greater flexibility where necessary






  • Database function called from SQL in SQL*Plus (DBFUNC)
  • Database function called from SQL in PL/SQL (DBFUNC)
  • Object constructor call (TABLE_COUNT_TYPE)

Call Structure Diagram
HProf - CSD

Raw Results
The attached script Test_Rep_h.sql was used to report on the results. The record produced in the run table, DBMSHP_RUNS, was:

     RUNID RUN_TIMESTAMP                   MICRO_S    SECONDS RUN_COMMENT
---------- ---------------------------- ---------- ---------- ------------------------------------------------------------
        11 04-MAR-13 07.07.36.803000        890719        .89 Profile for small test program with recursion

The records produced in the functions table, DBMSHP_FUNCTION_INFO, were:

OWNER MODULE               FUNCTION                         ID  LINE#      SUB_T      FUN_T  CALLS
----- -------------------- ------------------------------ ---- ------ ---------- ---------- ------
NET   HPROF_TEST           A_CALLS_B                         4     40      62340       4450      1
NET   HPROF_TEST           A_CALLS_B@1                       5     40      43729      13663      1
NET   HPROF_TEST           B_CALLS_A                         6     38      57890      14161      1
NET   HPROF_TEST           B_CALLS_A@1                       7     38      30066      30066      1
NET   HPROF_TEST           DBFUNC                            8     84      32629      32629      2
NET   HPROF_TEST           R_CALLS_R                         9     70      12823       4159      1
NET   HPROF_TEST           R_CALLS_R@1                      10     70       8633       8618      1
NET   HPROF_TEST           STOP_PROFILING                   11     16         21         21      1
NET   TABLE_COUNT_TYPE     TABLE_COUNT_TYPE                 12      3      55049         82      1
NET   TABLE_COUNT_TYPE     __static_sql_exec_line6          22      6      54967      54967      1
SYS   DBMS_HPROF           STOP_PROFILING                   13     59          0          0      1
SYS   DBMS_OUTPUT          GET_LINE                         14    129          8          8      3
SYS   DBMS_OUTPUT          GET_LINES                        15    160         68         60      3
SYS   DBMS_OUTPUT          NEW_LINE                         16    117          7          7      2
SYS   DBMS_OUTPUT          PUT                              17     77         28         28      2
SYS   DBMS_OUTPUT          PUT_LINE                         18    109         46         11      2
                           __anonymous_block                 1      0     809839        521      5
                           __dyn_sql_exec_line12            19     12        226        226      1
                           __plsql_vm                        2      0     828379         58      6
                           __plsql_vm@1                      3      0      14158         11      1
                           __sql_fetch_line13               20     13     726713     726713      1
                           __static_sql_exec_line8          21      8      14418        260      1

22 rows selected.

The SUB_T and FUN_T values are the total times in microseconds for the subtree including function, and function-only processing.

The records produced in the functions parent-child table, DBMSHP_PARENT_CHILD_INFO, were:

OWNER_P MODULE_P             FUNCTION_P                     OWNER_C MODULE_C             FUNCTION_C                          SUB_T      FUN_T  CALLS
------- -------------------- ------------------------------ ------- -------------------- ------------------------------ ---------- ---------- ------
NET     HPROF_TEST           STOP_PROFILING                 SYS     DBMS_HPROF           STOP_PROFILING                          0          0      1
NET     HPROF_TEST           R_CALLS_R@1                    SYS     DBMS_OUTPUT          PUT_LINE                               15          6      1
NET     HPROF_TEST           R_CALLS_R                      SYS     DBMS_OUTPUT          PUT_LINE                               31          5      1
NET     HPROF_TEST           R_CALLS_R                      NET     HPROF_TEST           R_CALLS_R@1                          8633       8618      1
NET     HPROF_TEST           B_CALLS_A                      NET     HPROF_TEST           A_CALLS_B@1                         43729      13663      1
NET     HPROF_TEST           A_CALLS_B@1                    NET     HPROF_TEST           B_CALLS_A@1                         30066      30066      1
NET     HPROF_TEST           A_CALLS_B                      NET     HPROF_TEST           B_CALLS_A                           57890      14161      1
NET     TABLE_COUNT_TYPE     TABLE_COUNT_TYPE               NET     TABLE_COUNT_TYPE     __static_sql_exec_line6             54967      54967      1
SYS     DBMS_OUTPUT          PUT_LINE                       SYS     DBMS_OUTPUT          NEW_LINE                                7          7      2
SYS     DBMS_OUTPUT          PUT_LINE                       SYS     DBMS_OUTPUT          PUT                                    28         28      2
SYS     DBMS_OUTPUT          GET_LINES                      SYS     DBMS_OUTPUT          GET_LINE                                8          8      3
                             __anonymous_block              NET     HPROF_TEST           STOP_PROFILING                         21         21      1
                             __anonymous_block              SYS     DBMS_OUTPUT          GET_LINES                              68         60      3
                             __anonymous_block                                           __dyn_sql_exec_line12                 226        226      1
                             __anonymous_block              NET     HPROF_TEST           R_CALLS_R                           12823       4159      1
                             __anonymous_block                                           __static_sql_exec_line8             14418        260      1
                             __plsql_vm                     NET     HPROF_TEST           DBFUNC                              18482      18482      1
                             __anonymous_block                                           __sql_fetch_line13                 726713     726713      1
                             __static_sql_exec_line8                                     __plsql_vm@1                        14158         11      1
                             __plsql_vm                                                  __anonymous_block                  809839        521      5
                             __plsql_vm@1                   NET     HPROF_TEST           DBFUNC                              14147      14147      1
                             __anonymous_block              NET     TABLE_COUNT_TYPE     TABLE_COUNT_TYPE                    55049         82      1

22 rows selected.

The SUB_T and FUN_T values are the total times in microseconds for the subtree including function, and function-only processing, respectively, for the child function while called from all instances of the parent.

Function Call Tree
The raw data above can be used to identify processing bottlenecks at a function level, but it's also useful to process the data in order to display the function hierarchies, both for performance tuning and also for understanding the program structure. This is not quite as trivial as it may seem. The oracle-base article provides an SQL statement that attempts to do this:

SELECT RPAD(' ', level*2, ' ') || fi.owner || '.' || fi.module AS name,
       fi.function,
       pci.subtree_elapsed_time,
       pci.function_elapsed_time,
       pci.calls
FROM   dbmshp_parent_child_info pci
       JOIN dbmshp_function_info fi ON pci.runid = fi.runid AND pci.childsymid = fi.symbolid
WHERE  pci.runid = :RUN_ID
CONNECT BY PRIOR childsymid = parentsymid
START WITH pci.parentsymid = :START_ID

Here, bind variables replace the original hard-coded values. On running this query I often got the following result:

ERROR at line 1:
ORA-01436: CONNECT BY loop in user data

On the run used in this article, the query returned 157 records, which is obviously incorrect. There is of course a NOCYCLE keyword that can be used to return results in the case of loops. However, it is not worth adding in this case, because there are in fact no loops in the data (at least no cyclic loops - apparent loops are discussed later). Oracle avoids loops by treating a function call that is a descendant of itself as a call to a new function, identified by suffices @1, @2 etc. as we can see from the recursive procedures above (eg R_CALLS_R@1 is the second call of R_CALLS_R, this one from itself). The problem here is that the query is incorrect in its handling of runid, with the result that the tree-walk traverses records from other runs as well as the intended one. A further problem is that there may be several roots, and it would be best to calculate these within a subquery. We can correct these problems by the following query:

SELECT RPAD(' ', level*2, ' ') || fi.owner || '.' || fi.module AS name,
       fi.symbolid || ': ' || fi.function function,
       pci.subtree_elapsed_time sub_t,
       pci.function_elapsed_time fun_t,
       pci.calls
  FROM dbmshp_parent_child_info	pci
  JOIN dbmshp_function_info		fi 
    ON pci.runid	              = fi.runid 
   AND pci.childsymid	       = fi.symbolid
 WHERE pci.runid                   = :RUN_ID
CONNECT BY PRIOR pci.childsymid    = pci.parentsymid 
       AND pci.runid	              = :RUN_ID
START WITH pci.parentsymid         IN (SELECT f.symbolid FROM dbmshp_function_info f WHERE NOT EXISTS 
       (SELECT 1 FROM dbmshp_parent_child_info i WHERE i.childsymid = f.symbolid AND i.runid = :RUN_ID) AND f.runid = :RUN_ID)
       AND pci.runid	              = :RUN_ID

This query returns the results:

NAME                           FUNCTION                            SUB_T      FUN_T  CALLS
------------------------------ ------------------------------ ---------- ---------- ------
  .                            1: __anonymous_block              809,839        521      5
    NET.HPROF_TEST             9: R_CALLS_R                       12,823      4,159      1
      NET.HPROF_TEST           10: R_CALLS_R@1                     8,633      8,618      1
        SYS.DBMS_OUTPUT        18: PUT_LINE                           15          6      1
          SYS.DBMS_OUTPUT      16: NEW_LINE                            7          7      2
          SYS.DBMS_OUTPUT      17: PUT                                28         28      2
      SYS.DBMS_OUTPUT          18: PUT_LINE                           31          5      1
        SYS.DBMS_OUTPUT        16: NEW_LINE                            7          7      2
        SYS.DBMS_OUTPUT        17: PUT                                28         28      2
    NET.HPROF_TEST             11: STOP_PROFILING                     21         21      1
      SYS.DBMS_HPROF           13: STOP_PROFILING                      0          0      1
    NET.TABLE_COUNT_TYPE       12: TABLE_COUNT_TYPE               55,049         82      1
      NET.TABLE_COUNT_TYPE     22: __static_sql_exec_line6        54,967     54,967      1
    SYS.DBMS_OUTPUT            15: GET_LINES                          68         60      3
      SYS.DBMS_OUTPUT          14: GET_LINE                            8          8      3
    .                          19: __dyn_sql_exec_line12             226        226      1
    .                          20: __sql_fetch_line13            726,713    726,713      1
    .                          21: __static_sql_exec_line8        14,418        260      1
      .                        3: __plsql_vm@1                    14,158         11      1
        NET.HPROF_TEST         8: DBFUNC                          14,147     14,147      1
  NET.HPROF_TEST               8: DBFUNC                          18,482     18,482      1
  NET.HPROF_TEST               6: B_CALLS_A                       57,890     14,161      1
    NET.HPROF_TEST             5: A_CALLS_B@1                     43,729     13,663      1
      NET.HPROF_TEST           7: B_CALLS_A@1                     30,066     30,066      1

24 rows selected.

This is better, but we can identify some further issues.

Missing Roots
The true root results are missing: For example, A_CALLS_B is missing. This arises because the query is traversing the link records (DBMSHP_PARENT_CHILD_INFO), while the root information is stored in the nodes (DBMSHP_FUNCTION_INFO). This suggests a change from the CONNECT BY syntax to Oracle's v11.2 recursive subquery factoring syntax, which allows you easily to start from the nodes, then traverse recursively via the links. (Incidentally, moving the start of profiling to its own block would result in A_CALLS_B appearing under __anonymous_block, but I prefer to retain the current structure in order to deal with the general case in which multiple roots are possible.)

Duplicate Links
Notice that function PUT_LINE is reported separately under R_CALLS_R and R_CALLS_R@1, and the timings differ. Also, its own child calls appear under each of its instances, but in those cases the timings are identical. The reason for this is that in the first case, there are separate records of the times used in each call, whereas in the second, the child calls have only a single record giving the total times across both instances of the parent call. The call from R_CALLS_R shows (9 - 4 = ) 5µs used in child calls, while the call from R_CALLS_R@1 shows 14µs. The child calls show totals of (3 + 16 = ) 19µs, equalling the sum across the parent calls.

At this point it is worth looking at this from the more general perspective of a hierarchical data structure where parents can have multiple children and children multiple parents, with one or more roots. If a network diagram were constructed there would be loops apparent indicating multiple routes between nodes. In these situations, Oracle's hierarchical queries effectively traverse all routes, and this is what causes the link duplication (in other scenarios this behaviour can cause big performance problems, but probably not here). Oracle's cycle detection mechanism does not trigger because the loops do not result in any node being a descendant of itself (as noted above, extra nodes are generated by the profiler to avoid this).

It seems to me better to avoid this duplication, and also to signal those cases where times are not aggregated up the tree. We can achieve this by the use of analytic functions. Note that, although the query below refers to the specific tables and attributes for this problem, the proposed solution could be used for any member of this general class of problem. The new query, which orders sibling records by descending subtree elapsed time, is:

WITH last_run AS (
SELECT Max (runid) runid FROM dbmshp_runs
), full_tree (runid, lev, node_id, sub_t, fun_t, calls, link_id) AS (
SELECT fni.runid, 0, fni.symbolid, fni.subtree_elapsed_time, fni.function_elapsed_time, fni.calls, 'root' || ROWNUM
  FROM dbmshp_function_info fni
  JOIN last_run lrn
    ON lrn.runid = fni.runid
 WHERE NOT EXISTS (SELECT 1 FROM dbmshp_parent_child_info pci WHERE pci.childsymid = fni.symbolid AND pci.runid = fni.runid)
 UNION ALL
SELECT ftr.runid, 
       ftr.lev + 1, 
       pci.childsymid, 
       pci.subtree_elapsed_time, 
       pci.function_elapsed_time, 
       pci.calls,
       pci.parentsymid || '-' || pci.childsymid
  FROM full_tree ftr
  JOIN dbmshp_parent_child_info pci
    ON pci.parentsymid = ftr.node_id
   AND pci.runid = ftr.runid
) SEARCH DEPTH FIRST BY sub_t DESC, fun_t DESC, calls DESC, node_id SET rn
, tree_ranked AS (
SELECT runid, node_id, lev, rn, 
       sub_t, fun_t, calls, 
       Row_Number () OVER (PARTITION BY node_id ORDER BY rn) node_rn,
       Count (*) OVER (PARTITION BY node_id) node_cnt,
       Row_Number () OVER (PARTITION BY link_id ORDER BY rn) link_rn
  FROM full_tree
)
SELECT RPad (' ', trr.lev*2, ' ') || fni.function "Function tree",
       fni.symbolid sy, fni.owner, fni.module,
       CASE WHEN trr.node_cnt > 1 THEN trr.node_rn || ' of ' || trr.node_cnt END "Inst.",
       trr.sub_t, trr.fun_t, trr.calls, 
       trr.rn "Row"
  FROM tree_ranked trr
  JOIN dbmshp_function_info fni
    ON fni.symbolid = trr.node_id
   AND fni.runid = trr.runid
 WHERE trr.link_rn = 1
 ORDER BY trr.rn

Query Structure Diagram
HProf - QSD

The results are then:

Function tree                        SY OWNER MODULE               Inst.         SUB_T      FUN_T  CALLS  Row
----------------------------------- --- ----- -------------------- -------- ---------- ---------- ------ ----
__plsql_vm                            2                                        828,379         58      6    1
  __anonymous_block                   1                                        809,839        521      5    2
    __sql_fetch_line13               20                                        726,713    726,713      1    3
    TABLE_COUNT_TYPE                 12 NET   TABLE_COUNT_TYPE                  55,049         82      1    4
      __static_sql_exec_line6        22 NET   TABLE_COUNT_TYPE                  54,967     54,967      1    5
    __static_sql_exec_line8          21                                         14,418        260      1    6
      __plsql_vm@1                    3                                         14,158         11      1    7
        DBFUNC                        8 NET   HPROF_TEST           1 of 2       14,147     14,147      1    8
    R_CALLS_R                         9 NET   HPROF_TEST                        12,823      4,159      1    9
      R_CALLS_R@1                    10 NET   HPROF_TEST                         8,633      8,618      1   10
        PUT_LINE                     18 SYS   DBMS_OUTPUT          1 of 2           15          6      1   11
          PUT                        17 SYS   DBMS_OUTPUT          1 of 2           28         28      2   12
          NEW_LINE                   16 SYS   DBMS_OUTPUT          1 of 2            7          7      2   13
      PUT_LINE                       18 SYS   DBMS_OUTPUT          2 of 2           31          5      1   14
    __dyn_sql_exec_line12            19                                            226        226      1   17
    GET_LINES                        15 SYS   DBMS_OUTPUT                           68         60      3   18
      GET_LINE                       14 SYS   DBMS_OUTPUT                            8          8      3   19
    STOP_PROFILING                   11 NET   HPROF_TEST                            21         21      1   20
      STOP_PROFILING                 13 SYS   DBMS_HPROF                             0          0      1   21
  DBFUNC                              8 NET   HPROF_TEST           2 of 2       18,482     18,482      1   22
A_CALLS_B                             4 NET   HPROF_TEST                        62,340      4,450      1   23
  B_CALLS_A                           6 NET   HPROF_TEST                        57,890     14,161      1   24
    A_CALLS_B@1                       5 NET   HPROF_TEST                        43,729     13,663      1   25
      B_CALLS_A@1                     7 NET   HPROF_TEST                        30,066     30,066      1   26

24 rows selected.

Notice that we now have a single record for each of the 22 links, plus the two root nodes. Also, the "Inst." column lists the instance number of a function having more than one instance, and the children of any such function are only listed once with the gaps in the "Row" column indicating where duplicates have been suppressed.

Network Diagrams
It may be interesting to display the call tree in two diagrams, one for each root.
Root __plsql_vm
HProf - Net

Root A_CALLS_B
HProf - Net2

Notes on Tree Output
Anonymous Block (__anonymous_block)
This function seems to correspond to invocations of anonymous blocks, obviously enough. However, there is an apparent anomaly in the number of calls listed, 6, because the driving program has only three such blocks, and there are none in the called PL/SQL code. I would surmise that the apparent discrepancy arises from the enabling of SERVEROUTPUT, which appears to result in a secondary block being associated with each explicit SQL*Plus block, that issues a call to GET_LINES to process buffered output.

PL/SQL Engine (__plsql_vm)
This function seems to correspond to external invocations of PL/SQL such as from a SQL*Plus session. There are 7 calls, 6 of them presumably being linked with the external anonymous blocks, and the seventh with DBFUNC, where a PL/SQL function is called from a SQL statement from SQL*Plus.

Notice that the SQL statement calling a database function from within PL/SQL generates the recursive call to the engine, __plsql_vm@1

Second Root (A_CALLS_B)
The above function does not have the __plsql_vm/__anonymous_block ancestry that might be expected because profiling only started within the enclosing block.

Inlined Procedure (Rest_a_While)
I wrote a small procedure, Rest_a_While, to generate some elapsed time in the recursive procedures, but preceded it with the INLINE pragma, a new optimisation feature in 11g. This had the desired effect of removing the calls from the profiling output and including the times in the calling procedures. Rest_a_While does not make the obvious call to DBMS_Lock.Sleep because that procedure cannot be inlined. subprogram inlining in 11g provides some analysis of the inlining feature.

Sibling Ordering
We have ordered siblings by descending subtree elapsed time, using the SEARCH clause. It would be nice to have the option to order the siblings by initial invocation time, but Oracle does not provide the data to do this.

Loops and Hierarchies
The first diagram shows two loops, where there are two routes between the loop start and end points, indicated by different colours. The second loop has two child nodes coming from the end point, and hierarchical queries (both CONNECT BY and recursive subquery factors in Oracle) cause the links to be duplicated. Our query has filtered out the duplicates by analytic functions.

It's worth remembering this because it's a general feature of SQL for querying hierarchies, and judging by Oracle forums, not one that's widely understood. For larger hierarchies it can cause serious performance problems, and may justify a PL/SQL programmed solution that need not suffer the same problem.

Manual Instrumentation
Oracle's hierarchical profiler clearly provides extremely useful information on both performance and structure of PL/SQL programs with very little effort. However, it does have the limitation of only providing information down to the subprogram level (which includes embedded SQL statements in this context). It is also often considered good practice to implement timing and other instrumentation permanently in production code, sometimes in a switchable fashion. In the test program, one of the called procedures, A_Calls_B, makes two calls to the inlined procedure, Rest_a_While, the second doing about twice as much work as the first. The profiler reports total within-function times of 4,450µs and 13,663µs on first and second calls, respectively (the work is scaled by a call number parameter, equal to 1, then 3).

I created a second instance of the package and driver script (suffix _TS) to illustrate manual instrumentation. This uses an 'object-oriented' timing package that I wrote a couple of years ago Code Timing and Object Orientation and Zombies (November, 2010) to instrument at procedure and section level. I multiplied the work in Rest_a_While by a factor of ten to get larger times. This produced the output:

Timer Set: HProf, Constructed at 05 Mar 2013 10:21:27, written at 10:21:30
==========================================================================
[Timer timed: Elapsed (per call): 0.04 (0.000044), CPU (per call): 0.05 (0.000050), calls: 1000, '***' denotes corrected line below]

Timer                       Elapsed          CPU          Calls        Ela/Call        CPU/Call
----------------------   ----------   ----------   ------------   -------------   -------------
A_Calls_B, section one         0.06         0.05              2         0.03150         0.02500
A_Calls_B, section two         0.12         0.12              2         0.06050         0.06000
B_Calls_A: 2                   0.15         0.16              1         0.15400         0.16000
B_Calls_A: 4                   0.31         0.30              1         0.30700         0.30000
DBFunc                         0.32         0.31              2         0.15950         0.15500
Open cursor                    0.69         0.69              1         0.68900         0.69000
Fetch from cursor              0.70         0.70              1         0.69600         0.70000
Close cursor                   0.00         0.00              1         0.00000         0.00000
Construct object               0.06         0.04              1         0.05500         0.04000
R_Calls_R                      0.14         0.14              2         0.07000         0.07000
(Other)                        0.00         0.00              1         0.00000         0.00000
----------------------   ----------   ----------   ------------   -------------   -------------
Total                          2.54         2.51             15         0.16960         0.16733
----------------------   ----------   ----------   ------------   -------------   -------------

Notes on Code Timing

  • Calls, CPU and elapsed times have been captured at the section level for A_Calls_B
  • Observe that, while R_Calls_R and A_Calls_B aggregate over all calls, B_Calls_A records values by call; this is implemented simply by including a value that changes with call in the timer name
  • The timing set object is designed to be very low footprint; here 9 statements (calls to Increment_Time), plus a small global overhead, produced 10 result lines, plus associated information
  • The 'object-oriented' approach allows multiple programs to be be timed at multiple levels, without interference between timings
  • There are Perl and Java implementations of this timing set object included in the Scribd article mentioned

Oracle's Flat Profiler (DBMS_Profiler)
The hierarchical profiler was introduced in v11.1, while prior to this there was a non-hierarchical profiler, DBMS_Profiler. This package still exists in v11: It is omitted from the advanced application developer's guide for v11, but is described in the packages and types manual (Oracle® Database PL/SQL Packages and Types Reference, 11g Release 2 (11.2)); also, SQL*Developer appears to support only the newer hierarchical verion (via right-click on a package). I thought it interesting to run the older version on the same test program (package Old_Test_Prof, driver script Test_Rep_p_Old.sql and reporting script Test_Rep_h_Old.sql). The output from the first three queries is:

Run header (PLSQL_PROFILER_RUNS)

     RUNID RUN_DATE        MICRO_S    SECONDS
---------- ------------ ---------- ----------
         3 11:03:13        2164000       2.16

Profiler data summary (PLSQL_PROFILER_DATA)

   MICRO_S SECONDS    CALLS
---------- ------- --------
   2126949    2.13       72

Profiler data by time (PLSQL_PROFILER_DATA)

   MICRO_S SECONDS    CALLS UNIT_NAME            UNIT_NUMBER  LINE#
---------- ------- -------- -------------------- ----------- ------
    729932    0.73        1                                5     13
    569563    0.57        2 OLD_PROF_TEST                  1     56
    377880    0.38        2 OLD_PROF_TEST                  1     82
    166019    0.17        2 OLD_PROF_TEST                  1     70
    150117    0.15        2 OLD_PROF_TEST                  1     43
     72742    0.07        2 OLD_PROF_TEST                  1     40
     56473    0.06        1 TABLE_COUNT_TYPE               6      6
      3338    0.00        1                                5      8
       258    0.00        1                                5     12
       109    0.00        1                                5     16
        68    0.00        2 OLD_PROF_TEST                  1     67
        66    0.00        2                                4      1
        60    0.00        2                                7      1
        60    0.00        2                                3      1
        44    0.00        1                                5     14
        42    0.00        0                                2      5
        31    0.00        1 OLD_PROF_TEST                  1     18
        26    0.00        1                                8      5
        13    0.00        0                                5      1
         9    0.00        1 TABLE_COUNT_TYPE               6     11
         9    0.00        2 OLD_PROF_TEST                  1     86
         8    0.00        0 OLD_PROF_TEST                  1     51
         8    0.00        1 TABLE_COUNT_TYPE               6     13
         7    0.00        1                                5     18
         6    0.00        0 OLD_PROF_TEST                  1     78
         6    0.00        0 OLD_PROF_TEST                  1     64
         6    0.00        0                                8      1
         6    0.00        1 TABLE_COUNT_TYPE               6      3
         5    0.00        0 OLD_PROF_TEST                  1     35
         5    0.00        0 OLD_PROF_TEST                  1     15
         4    0.00        1                                8      7
         4    0.00        1 OLD_PROF_TEST                  1     76
         3    0.00        1                                2      8
         2    0.00        1 OLD_PROF_TEST                  1     62
         2    0.00        1 OLD_PROF_TEST                  1     13
         2    0.00        1 TABLE_COUNT_TYPE               6      5
         2    0.00        2 OLD_PROF_TEST                  1     72
         2    0.00        2 OLD_PROF_TEST                  1     45
         2    0.00        2 OLD_PROF_TEST                  1     49
         2    0.00        2 OLD_PROF_TEST                  1     46
         2    0.00        2 OLD_PROF_TEST                  1     58
         1    0.00        1 OLD_PROF_TEST                  1     73
         1    0.00        1                                2      6
         1    0.00        1 OLD_PROF_TEST                  1     59
         1    0.00        1 OLD_PROF_TEST                  1     11
         1    0.00        2 OLD_PROF_TEST                  1     54
         1    0.00        2 OLD_PROF_TEST                  1     84
         0    0.00        0 OLD_PROF_TEST                  1      1
         0    0.00        0 OLD_PROF_TEST                  1     88
         0    0.00        0                                8      9
         0    0.00        0                                2      1
         0    0.00        0                                2      2
         0    0.00        0 OLD_PROF_TEST                  1      3
         0    0.00        0 OLD_PROF_TEST                  1      5
         0    0.00        0 OLD_PROF_TEST                  1      9
         0    0.00        0 OLD_PROF_TEST                  1     20
         0    0.00        1 TABLE_COUNT_TYPE               6      4
         0    0.00        1                                8      2
         0    0.00        2 OLD_PROF_TEST                  1     39
         0    0.00        2 OLD_PROF_TEST                  1     55
         0    0.00        2 OLD_PROF_TEST                  1     69
         0    0.00        2 OLD_PROF_TEST                  1     38
         0    0.00        2 OLD_PROF_TEST                  1     81
         0    0.00        2 OLD_PROF_TEST                  1     42
         0    0.00        2 OLD_PROF_TEST                  1     68

65 rows selected.

Referring to the package, type and anonymous blocks, I assigned labels to all the lines having more than 10µs, as follows:

   MICRO_S SECONDS    CALLS UNIT_NAME            UNIT_NUMBER  LINE#
---------- ------- -------- -------------------- ----------- ------
    729932    0.73        1                                5     13  B2: FETCH
    569563    0.57        2 OLD_PROF_TEST                  1     56  B_Calls_A (Rest_a_While)
    377880    0.38        2 OLD_PROF_TEST                  1     82  DBFunc (Rest_a_While)
    166019    0.17        2 OLD_PROF_TEST                  1     70  R_Calls_R (Rest_a_While)
    150117    0.15        2 OLD_PROF_TEST                  1     43  A_Calls_B (Rest_a_While, section 2)
     72742    0.07        2 OLD_PROF_TEST                  1     40  A_Calls_B (Rest_a_While, section 1)
     56473    0.06        1 TABLE_COUNT_TYPE               6      6  SELECT
      3338    0.00        1                                5      8  B2: SELECT DBFunc
       258    0.00        1                                5     12  B2: OPEN
       109    0.00        1                                5     16  B2: Assign Table_Count_Type
        68    0.00        2 OLD_PROF_TEST                  1     67  Put_Line
        66    0.00        2                                4      1  Auxiliary SERVEROUTPUT block for B2 (surmised)
        60    0.00        2                                7      1  Auxiliary SERVEROUTPUT block for B3 (surmised)
        60    0.00        2                                3      1  Auxiliary SERVEROUTPUT block for B1 (surmised)
        44    0.00        1                                5     14  B2: CLOSE
        42    0.00        0                                2      5  B1: Call to Start_Profiling 
        31    0.00        1 OLD_PROF_TEST                  1     18  RETURN DBMS_Profiler.Stop_Profiler;
        26    0.00        1                                8      5  B3: Call R_Calls_R
        13    0.00        0                                5      1  B2: DECLARE

Notes on Output of Flat Profiler
There were six units with no linked information in DBMS_PROFILER_UNITS. By examining the data, I was able to associate unit numbers 2, 5 and 8 with my anonymous blocks B1, B2 and B3. That left three unassigned, and I have surmised that these correspond to the auxiliary blocks associated with processing server output that we earlier surmised when examining the output from the hierarchical profiler.

  • The useful call tree structure is not present in the data from the old profiler
  • However, the results are at a line level, which the hierarchical profiler does not provide; for example, the two sections of A_Calls_B are reported separately
  • Deciphering the output requires significantly more manual effort than with the hierarchical profiler
  • Both old and new profiler have their own advantages, and so both should be considered of value
  • Manual code timing offers more flexibility in terms of aggregating lines and call instances, but requires more effort...
  • ...but not as much as I thought. As noted later on the second example, after reading another article on the profiler, I realised that I could join the system table ALL_SOURCE to see the text of the line (where available)

Second example: Flat profiler omits some detail timings
After posting the first draft of this article, which was about the newer hierarchical profiler only, I noticed a new post on an old AskTom thread on the older flat profiler. The post concerned a discrepancy between reported times at the aggregate level and detail levels. I suggested using the hierarchical profiler might resolve the problem Try the hierarchical profiler..., and then added sections on the old profiler and on manual timing to this article for comparison. However, my example programs above do not include the AskTom scenario, so I later decided to add a new small scenario to illustrate it and now report the results here. The new test code consists of a PL/SQL block with two calls to DBMS_Lock.Sleep, for 3 and 6 seconds, with the profiling code around them. The driving scripts and output files are included in the zip file attached, and I list summary results below:

I later came upon another artilce on the flat profiler, Profiling PL/SQL with dbms_profiler where the author has joined the system table ALL_SOURCE to get the text of the line profiled, which makes interpretation easier. I have then updated the line-level query as follows:

PROMPT Profiler data by time (PLSQL_PROFILER_DATA)
SELECT Round (dat.total_time/1000, 0)  micro_s,
       Round (dat.total_time/1000000000, 2) seconds,
       dat.total_occur calls,
       unt.unit_name,
       dat.unit_number,
       dat.line#,
       Trim (src.text) text
  FROM plsql_profiler_data dat
  LEFT JOIN plsql_profiler_units unt
    ON unt.runid            = dat.runid
   AND unt.unit_number      = dat.unit_number
  LEFT JOIN all_source      src
    ON src.type             IN ('PACKAGE BODY','FUNCTION','PROCEDURE','TRIGGER')
   AND src.name             = unt.unit_name 
   AND src.line             = dat.line# 
   AND src.owner            = unt.unit_owner 
   AND src.type             = unt.unit_type
 WHERE dat.runid            = :runid
   AND dat.total_time       > 0
 ORDER BY 1 DESC, 2, 3

Of course the text is only available for stored source, so excludes lines from anonymous blocks.

Flat Profiler

Run header (PLSQL_PROFILER_RUNS)

     RUNID RUN_DATE        MICRO_S    SECONDS
---------- ------------ ---------- ----------
         5 20:34:45        9220000       9.22

Profiler data by unit (PLSQL_PROFILER_DATA)

UNIT_NAME            UNIT_NUMBER    MICRO_S SECONDS    CALLS
-------------------- ----------- ---------- ------- --------
                               2        200    0.00        3
UTILS                          1         30    0.00        3

Profiler data by time (PLSQL_PROFILER_DATA)

   MICRO_S SECONDS    CALLS UNIT_NAME            UNIT_NUMBER  LINE# TEXT
---------- ------- -------- -------------------- ----------- ------ ------------------------------------------------
       136    0.00        1                     2      7
        21    0.00        1                     2     10
        19    0.00        1                     2     14
        14    0.00        0 UTILS                          1    343 FUNCTION Stop_D_Profiling RETURN PLS_INTEGER IS
        13    0.00        1 UTILS                          1    346 RETURN DBMS_Profiler.Stop_Profiler;
         6    0.00        1 UTILS                          1    341 END Start_D_Profiling;
         2    0.00        1 UTILS                          1    339 RETURN l_run_number;

7 rows selected.

Hierarchical Profiler

Run header (DBMSHP_RUNS)

     RUNID RUN_TIMESTAMP                   MICRO_S    SECONDS RUN_COMMENT
---------- ---------------------------- ---------- ---------- -----------------------------
        16 19-MAR-13 21.37.00.571000       9000292          9 Profile for DBMS_Lock.Sleep

Functions called (DBMSHP_FUNCTION_INFO)

OWNER      MODULE               FUNCTION                LINE#      SUB_T      FUN_T  CALLS
---------- -------------------- ---------------------- ------ ---------- ---------- ------
BRENDAN    UTILS                1: STOP_H_PROFILING       322          8          8      1
SYS        DBMS_HPROF           2: STOP_PROFILING          59          0          0      1
SYS        DBMS_LOCK            3: SLEEP                  197    9000279    9000279      2
SYS        DBMS_LOCK            4: __pkg_init               0          5          5      1

Function call parent-child links (DBMSHP_PARENT_CHILD_INFO)

OWNER_P MODULE_P  FUNCTION_P            OWNER_C MODULE_C    FUNCTION_C         SUB_T FUN_T  CALLS
------- --------- --------------------- ------- ----------- ------------------ ----- ----- ------
BRENDAN UTILS     1: STOP_H_PROFILING   SYS     DBMS_HPROF  2: STOP_PROFILING      0     0      1

BPF Recursive Subquery Factor Tree Query

Function tree                              SUB_T      FUN_T  CALLS  Row
------------------------------------- ---------- ---------- ------ ----
3: SYS.DBMS_LOCK.SLEEP                   9000279    9000279      2    1
1: BRENDAN.UTILS.STOP_H_PROFILING              8          8      1    2
  2: SYS.DBMS_HPROF.STOP_PROFILING             0          0      1    3
4: SYS.DBMS_LOCK.__pkg_init                    5          5      1    4

Manual Profiler

Timer Set: Profiling DBMS_Lock.Sleep, Constructed at 19 Mar 2013 21:38:54, written at 21:39:03
==============================================================================================
[Timer timed: Elapsed (per call): 0.05 (0.000045), CPU (per call): 0.04 (0.000040), calls: 1000, '***' denotes corrected line below]

Timer               Elapsed          CPU          Calls        Ela/Call        CPU/Call
--------------   ----------   ----------   ------------   -------------   -------------
3 second sleep         3.00         0.00              1         3.00000         0.00000
6 second sleep         6.00         0.00              1         6.00100         0.00000
(Other)                0.00         0.00              1         0.00000         0.00000
--------------   ----------   ----------   ------------   -------------   -------------
Total                  9.00         0.00              3         3.00033         0.00000
--------------   ----------   ----------   ------------   -------------   -------------

Notes on Results for Second Example

  • The flat profiler shows 9s at header level but only 230µs at detail level because DBMS_Lock.Sleep does not permit profiling by the user running the script
  • The hierarchical profiler shows 9s at header level and a total of 9s in 2 calls to DBMS_Lock.Sleep
  • Manual profiling shows the two calls to DBMS_Lock.Sleep taking 3 and 6 seconds

Conclusions

  • Running Oracle's hierarchical profiler would seem to be the default first step in tuning PL/SQL programs from v11.1
  • Some care is needed in interpreting the output data; I've provided a query for displaying the hierarchies
  • Performance is recorded only down to function level, so it will still often be worthwhile to use the old flat profiler in addition
  • Manually timing code sections also still has a part to play, in terms of instrumentation and greater flexibility where necessary

Brendan HProf Code
Example 2






An SQL Solution for the Multiple Knapsack Problem (SKP-m)

In my last article, A Simple SQL Solution for the Knapsack Problem (SKP-1), I presented an SQL solution for the well known knapsack problem in its simpler 1-knapsack form (and it is advisable to read the first article before this one). Here I present an SQL solution for the problem in its more difficult multiple-knapsack form. The solution is a modified version of one I posted on OTN, SQL Query for mapping a set of batches to a class rooms group, and I describe two versions of it, one in pure SQL, and another that includes a database function. The earlier article provided the solutions as comma-separated strings of item identifiers, and in this article also the solutions are first obtained as delimited strings. However, as there are now containers as well as items, we extend the SQL to provide solutions with item and container names in separate fields within records for each container-item pair. The solution is presented, as before, more for its theoretical interest than for practical applicability. Much research has been done on procedural algorithms for this important, but computationally difficult class of problems.

We will consider the same simple example problem as in the earlier article, having four items, but now with two containers with individual weight limits of 8 and 10. As noted in the earlier article, the problem can be considered as that of assigning each item to one of the containers, or to none, leading directly to the expression for the number of not necessarily feasible assignment sets for the example. We can again depict the 24 possible item combinations in a diagram, with the container limits added.

Multi, v1.1 - CombisWe can see that there is one optimal solution in this case, in which items 1 and 3 are assigned to container 1, while items 2 and 4 are assigned to container 2, with a profit of 100. How to find it using SQL?

SQL Solution
The solution to the single knapsack problem worked by joining items recursively in increasing order of item id, accumulating the total weights and profits, and terminating a sequence when no more items can be added within the weight limit. The item sequences were accumulated as comma-separated strings, and the optimal solutions obtained by analytic ranking of the profits.

For the multiple knapsack problem, it's not quite as simple, but a similar approach may be a good starting point. Previously our anchor branch in the recursion selected all items below the single maximum weight, but we now have containers with individual weights. If we now join the containers table we can find all items falling within the maximum weights by container. The recursion can then proceed to find all feasible item combinations by container. Here is the SQL for this:

WITH rsf_itm (con_id, max_weight, itm_id, lev, tot_weight, tot_profit, path) AS (
SELECT c.id, 
       c.max_weight,
       i.id,
       0,
       i.item_weight,
       i.item_profit, 
       ',' || i.id || ','
  FROM items i
  JOIN containers c
    ON i.item_weight <= c.max_weight
 UNION ALL
SELECT r.con_id,
       r.max_weight,
       i.id, 
       r.lev + 1, 
       r.tot_weight + i.item_weight,
       r.tot_profit + i.item_profit,
       r.path || i.id || ','
  FROM rsf_itm r
  JOIN items i
    ON i.id > r.itm_id
   AND r.tot_weight + i.item_weight <= r.max_weight
 ORDER BY 1, 2
) SEARCH DEPTH FIRST BY con_id, itm_id SET line_no
SELECT con_id,
       max_weight,
       LPad (To_Char(itm_id), 2*lev + 1, ' ') itm_id, 
       path itm_path,
       tot_weight, tot_profit
  FROM rsf_itm
 ORDER BY line_no

and here is the resulting output:

CON_ID MAX_WEIGHT ITM_ID ITM_PATH    TOT_WEIGHT TOT_PROFIT
------ ---------- ------ ----------- ---------- ----------
     1          8 1      ,1,                  3         10
                    2    ,1,2,                7         30
                    3    ,1,3,                8         40
                  2      ,2,                  4         20
                  3      ,3,                  5         30
                  4      ,4,                  6         40
     2         10 1      ,1,                  3         10
                    2    ,1,2,                7         30
                    3    ,1,3,                8         40
                    4    ,1,4,                9         50
                  2      ,2,                  4         20
                    3    ,2,3,                9         50
                    4    ,2,4,               10         60
                  3      ,3,                  5         30
                  4      ,4,                  6         40

15 rows selected.

Looking at this, we can see that the overall solution will comprise one feasible combination of items for each container, with the constraint that no item appears in more than one container. This suggests that we could perform a second recursion in a similar way to the first, but this time using the results of the first as input, and joining the feasible combinations of containers of higher id only. If we again accumulate the sequence in a delimited string, regular expression functionality could be used to avoid joining combinations with items already included. The following SQL does this recursion:

WITH rsf_itm (con_id, max_weight, itm_id, tot_weight, tot_profit, path) AS (
SELECT c.id, 
       c.max_weight,
       i.id, 
       i.item_weight,
       i.item_profit, 
       ',' || i.id || ','
  FROM items i
  JOIN containers c
    ON i.item_weight <= c.max_weight
 UNION ALL
SELECT r.con_id,
       r.max_weight,
       i.id, 
       r.tot_weight + i.item_weight,
       r.tot_profit + i.item_profit,
       r.path || i.id || ','
  FROM rsf_itm r
  JOIN items i
    ON i.id > r.itm_id
   AND r.tot_weight + i.item_weight <= r.max_weight
)
, rsf_con (con_id, con_itm_set, con_itm_path, lev, tot_weight, tot_profit) AS (
SELECT con_id,
       ':' || con_id || ':' || path,
       ':' || con_id || ':' || path,
       0,
       tot_weight,
       tot_profit
  FROM rsf_itm
 UNION ALL
SELECT r_i.con_id,
       ':' || r_i.con_id || ':' || r_i.path,
       r_c.con_itm_path ||  ':' || r_i.con_id || ':' || r_i.path,
       r_c.lev + 1, 
       r_c.tot_weight + r_i.tot_weight,
       r_c.tot_profit + r_i.tot_profit
  FROM rsf_con r_c
  JOIN rsf_itm r_i
    ON r_i.con_id > r_c.con_id
 WHERE RegExp_Instr (r_c.con_itm_path || r_i.path, ',(\d+),.*?,\1,') = 0
) SEARCH DEPTH FIRST BY con_id SET line_no
SELECT
       LPad (' ', 2*lev, ' ') || con_itm_set con_itm_set,
       con_itm_path,
       tot_weight, tot_profit
  FROM rsf_con
 ORDER BY line_no

Notice the use of RegExp_Instr, which takes the current sequence with potential new combination appended as its source string, and looks for a match against the search string ',(\d+),.*?,\1,'. The function returns 0 if no match is found, meaning no duplicate item was found. The sequence includes the container id using a different delimiter, a colon, at the start of each combination. The search string can be explained as follows:

,(\d+), = a sequence of one or more digits with a comma either side, and the digit sequence saved for referencing
.*?,\1, = a sequence of any characters, followed by the saved digit sequence within commas. The ? specifies a non-greedy search, meaning stop searching as soon as a match is found

The result of the query is:

CON_ITM_SET          CON_ITM_PATH         TOT_WEIGHT TOT_PROFIT
-------------------- -------------------- ---------- ----------
:1:,1,               :1:,1,                        3         10
  :2:,2,             :1:,1,:2:,2,                  7         30
  :2:,3,             :1:,1,:2:,3,                  8         40
  :2:,4,             :1:,1,:2:,4,                  9         50
  :2:,2,3,           :1:,1,:2:,2,3,               12         60
  :2:,2,4,           :1:,1,:2:,2,4,               13         70
:1:,2,               :1:,2,                        4         20
  :2:,1,             :1:,2,:2:,1,                  7         30
  :2:,3,             :1:,2,:2:,3,                  9         50
  :2:,1,3,           :1:,2,:2:,1,3,               12         60
  :2:,4,             :1:,2,:2:,4,                 10         60
  :2:,1,4,           :1:,2,:2:,1,4,               13         70
:1:,1,2,             :1:,1,2,                      7         30
  :2:,3,             :1:,1,2,:2:,3,               12         60
  :2:,4,             :1:,1,2,:2:,4,               13         70
:1:,3,               :1:,3,                        5         30
  :2:,1,             :1:,3,:2:,1,                  8         40
  :2:,2,             :1:,3,:2:,2,                  9         50
  :2:,1,2,           :1:,3,:2:,1,2,               12         60
  :2:,4,             :1:,3,:2:,4,                 11         70
  :2:,1,4,           :1:,3,:2:,1,4,               14         80
  :2:,2,4,           :1:,3,:2:,2,4,               15         90
:1:,1,3,             :1:,1,3,                      8         40
  :2:,2,             :1:,1,3,:2:,2,               12         60
  :2:,4,             :1:,1,3,:2:,4,               14         80
  :2:,2,4,           :1:,1,3,:2:,2,4,             18        100
:1:,4,               :1:,4,                        6         40
  :2:,1,             :1:,4,:2:,1,                  9         50
  :2:,2,             :1:,4,:2:,2,                 10         60
  :2:,1,2,           :1:,4,:2:,1,2,               13         70
  :2:,3,             :1:,4,:2:,3,                 11         70
  :2:,1,3,           :1:,4,:2:,1,3,               14         80
  :2:,2,3,           :1:,4,:2:,2,3,               15         90
:2:,1,               :2:,1,                        3         10
:2:,2,               :2:,2,                        4         20
:2:,1,2,             :2:,1,2,                      7         30
:2:,3,               :2:,3,                        5         30
:2:,1,3,             :2:,1,3,                      8         40
:2:,4,               :2:,4,                        6         40
:2:,1,4,             :2:,1,4,                      9         50
:2:,2,3,             :2:,2,3,                      9         50
:2:,2,4,             :2:,2,4,                     10         60

42 rows selected.

We can see that the optimal solutions can be obtained from the output again using analytic ranking by profit, and in this case the solution with a profit of 100 is the optimal one, with sequence ':1:,1,3,:2:,2,4,'. In the full solution, as well as selecting out the top-ranking solutions, we have extended the query to output the items and containers by name, in distinct fields with a record for every solution/container/item combination. For the example problem above, the output is:

    SOL_ID S_WT  S_PR  C_ID C_NAME          M_WT C_WT  I_ID I_NAME     I_WT I_PR
---------- ---- ----- ----- --------------- ---- ---- ----- ---------- ---- ----
         1   18   100     1 Item 1             8    8     1 Item 1        3   10
                                                          3 Item 3        5   30
                          2 Item 2            10   10     2 Item 2        4   20
                                                          4 Item 4        6   40

SQL-Only Solution - XSQL
There are various techniques in SQL for splitting string columns into multiple rows and columns. We will take one of the more straightforward ones that uses the DUAL table with CONNECT BY to generate rows against which to anchor the string-parsing.

WITH rsf_itm (con_id, max_weight, itm_id, lev, tot_weight, tot_profit, path) AS (
SELECT c.id, 
       c.max_weight,
       i.id, 
       0, 
       i.item_weight,
       i.item_profit, 
       ',' || i.id || ','
  FROM items i
  JOIN containers c
    ON i.item_weight <= c.max_weight
 UNION ALL
SELECT r.con_id,
       r.max_weight,
       i.id, 
       r.lev + 1, 
       r.tot_weight + i.item_weight,
       r.tot_profit + i.item_profit,
       r.path || i.id || ','
  FROM rsf_itm r
  JOIN items i
    ON i.id > r.itm_id
   AND r.tot_weight + i.item_weight <= r.max_weight
)
, rsf_con (con_id, con_path, itm_path, tot_weight, tot_profit, lev) AS (
SELECT con_id,
       To_Char(con_id),
       ':' || con_id || '-' || (lev + 1) || ':' || path,
       tot_weight,
       tot_profit,
       0
  FROM rsf_itm
 UNION ALL
SELECT r_i.con_id,
       r_c.con_path || ',' || r_i.con_id,
       r_c.itm_path ||  ':' || r_i.con_id || '-' || (r_i.lev + 1) || ':' || r_i.path,
       r_c.tot_weight + r_i.tot_weight,
       r_c.tot_profit + r_i.tot_profit,
       r_c.lev + 1
  FROM rsf_con r_c
  JOIN rsf_itm r_i
    ON r_i.con_id > r_c.con_id
   AND RegExp_Instr (r_c.itm_path || r_i.path, ',(\d+),.*?,\1,') = 0
)
, paths_ranked AS (
SELECT itm_path || ':' itm_path, tot_weight, tot_profit, lev + 1 n_cons,
       Rank () OVER (ORDER BY tot_profit DESC) rnk
  FROM rsf_con
), best_paths AS (
SELECT itm_path, tot_weight, tot_profit, n_cons,
       Row_Number () OVER (ORDER BY tot_weight DESC) sol_id
  FROM paths_ranked
 WHERE rnk = 1
), row_gen AS (
SELECT LEVEL lev
  FROM DUAL
CONNECT BY LEVEL <= (SELECT Count(*) FROM items)
), con_v AS (
SELECT  r.lev con_ind, b.sol_id, b.tot_weight, b.tot_profit,
        Substr (b.itm_path, Instr (b.itm_path, ':', 1, 2*r.lev - 1) + 1, 
                            Instr (b.itm_path, ':', 1, 2*r.lev) - Instr (b.itm_path, ':', 1, 2*r.lev - 1) - 1)
           con_nit_id,
        Substr (b.itm_path, Instr (b.itm_path, ':', 1, 2*r.lev) + 1, 
                            Instr (b.itm_path, ':', 1, 2*r.lev + 1) - Instr (b.itm_path, ':', 1, 2*r.lev) - 1)
           itm_str
  FROM best_paths b
  JOIN row_gen r
    ON r.lev <= b.n_cons
), con_split AS (
SELECT sol_id, tot_weight, tot_profit,
       Substr (con_nit_id, 1, Instr (con_nit_id, '-', 1) - 1) con_id,
       Substr (con_nit_id, Instr (con_nit_id, '-', 1) + 1) n_items,
       itm_str
  FROM con_v
), itm_v AS (
SELECT  c.sol_id, c.con_id, c.tot_weight, c.tot_profit,
        Substr (c.itm_str, Instr (c.itm_str, ',', 1, r.lev) + 1, 
                            Instr (c.itm_str, ',', 1, r.lev + 1) - Instr (c.itm_str, ',', 1, r.lev) - 1)
           itm_id
  FROM con_split c
  JOIN row_gen r
    ON r.lev <= c.n_items
)
SELECT 
/* SEL */
       v.sol_id sol_id,
       v.tot_weight s_wt, 
       v.tot_profit s_pr, 
       c.id c_id, 
       c.name c_name, 
       c.max_weight m_wt,
       Sum (i.item_weight) OVER (PARTITION BY v.sol_id, c.id) c_wt,
       i.id i_id, 
       i.name i_name, 
       i.item_weight i_wt, 
       i.item_profit i_pr
/* SEL */
  FROM itm_v v
  JOIN containers c
    ON c.id = To_Number (v.con_id)
  JOIN items i
    ON i.id = To_Number (v.itm_id)
 ORDER BY sol_id, con_id, itm_id

SQL with Function Solution - XFUN
The SQL techniques for string-splitting are quite cumbersome, and a better approach may be the use of a pipelined function that allows the string-parsing to be done in PL/SQL, a procedural language that is better suited to the task.

WITH rsf_itm (con_id, max_weight, itm_id, tot_weight, tot_profit, path) AS (
SELECT c.id, 
       c.max_weight,
       i.id, 
       i.item_weight,
       i.item_profit, 
       ',' || i.id || ','
  FROM items i
  JOIN containers c
    ON i.item_weight <= c.max_weight
 UNION ALL
SELECT r.con_id,
       r.max_weight,
       i.id, 
       r.tot_weight + i.item_weight,
       r.tot_profit + i.item_profit,
       r.path || i.id || ','
  FROM rsf_itm r
  JOIN items i
    ON i.id > r.itm_id
   AND r.tot_weight + i.item_weight <= r.max_weight
 ORDER BY 1, 2
)
, rsf_con (con_id, itm_path, tot_weight, tot_profit) AS (
SELECT con_id,
       ':' || con_id || ':' || path,
       tot_weight,
       tot_profit
  FROM rsf_itm
 UNION ALL
SELECT r_i.con_id,
       r_c.itm_path ||  ':' || r_i.con_id || ':' || r_i.path,
       r_c.tot_weight + r_i.tot_weight,
       r_c.tot_profit + r_i.tot_profit
  FROM rsf_con r_c
  JOIN rsf_itm r_i
    ON r_i.con_id > r_c.con_id
   AND RegExp_Instr (r_c.itm_path || r_i.path, ',(\d+),.*?,\1,') = 0
)
, paths_ranked AS (
SELECT itm_path || ':' itm_path, tot_weight, tot_profit, Rank () OVER (ORDER BY tot_profit DESC) rn,
       Row_Number () OVER (ORDER BY tot_profit DESC, tot_weight DESC) sol_id
  FROM rsf_con
), itm_v AS (
SELECT s.con_id, s.itm_id, p.itm_path, p.tot_weight, p.tot_profit, p.sol_id
  FROM paths_ranked p
 CROSS JOIN TABLE (Multi.Split_String (p.itm_path)) s
 WHERE rn = 1
)
SELECT v.sol_id sol_id,
       v.tot_weight s_wt, 
       v.tot_profit s_pr, 
       c.id c_id, 
       c.name c_name, 
       c.max_weight m_wt,
       Sum (i.item_weight) OVER (PARTITION BY v.sol_id, c.id) c_wt,
       i.id i_id, 
       i.name i_name, 
       i.item_weight i_wt, 
       i.item_profit i_pr
  FROM itm_v v
  JOIN containers c
    ON c.id = To_Number (v.con_id)
  JOIN items i
    ON i.id = To_Number (v.itm_id)
 ORDER BY sol_id, con_id, itm_id

Pipelined Database Function

CREATE OR REPLACE TYPE con_itm_type AS OBJECT (con_id NUMBER, itm_id NUMBER);
/
CREATE OR REPLACE TYPE con_itm_list_type AS VARRAY(100) OF con_itm_type;
/
CREATE OR REPLACE PACKAGE BODY Multi IS

FUNCTION Split_String (p_string VARCHAR2) RETURN con_itm_list_type PIPELINED IS

  l_pos_colon_1           PLS_INTEGER := 1;
  l_pos_colon_2           PLS_INTEGER;
  l_pos_comma_1           PLS_INTEGER;
  l_pos_comma_2           PLS_INTEGER;
  l_con                   PLS_INTEGER;
  l_itm                   PLS_INTEGER;

BEGIN

  LOOP

    l_pos_colon_2 := Instr (p_string, ':', l_pos_colon_1 + 1, 1);
    EXIT WHEN l_pos_colon_2 = 0;

    l_con := To_Number (Substr (p_string, l_pos_colon_1 + 1, l_pos_colon_2 - l_pos_colon_1 - 1));
    l_pos_colon_1 := Instr (p_string, ':', l_pos_colon_2 + 1, 1);
    l_pos_comma_1 := l_pos_colon_2 + 1;

    LOOP

      l_pos_comma_2 := Instr (p_string, ',', l_pos_comma_1 + 1, 1);
      EXIT WHEN l_pos_comma_2 = 0 OR l_pos_comma_2 > l_pos_colon_1;

      l_itm := To_Number (Substr (p_string, l_pos_comma_1 + 1, l_pos_comma_2 - l_pos_comma_1 - 1));
      PIPE ROW (con_itm_type (l_con, l_itm));
      l_pos_comma_1 := l_pos_comma_2;
 
    END LOOP;

  END LOOP;

END Split_String;

END Multi;

Query Structure Diagram (embedded directly)
The QSD shows both queries in a single diagram as the early query blocks are almost the same (the main difference is that the strings contain a bit more information for XSQL to facilitate the later splitting). The directly-embedded version shows the whole query, but it may be hard to read the detail, so it is followed by a larger, scrollable version within Excel.
QSD shwoing both versions of SQL

Query Structure Diagram (embedded via Excel)
This is the larger, scrollable version.

Performance Analysis
As in the previous article, we will see how the solution methods perform as problem size varies, using my own performance benchmarking framework.

Test Data Sets
Test data sets are generated as follows, in terms of two integer parameters, w and d:

  • Insert w containers with sequential ids and random maximum weights between 1 and 100
  • Insert d items with sequential ids and random weights and profits in the ranges 1-60 and 1-10000, respectively, via Oracle's function DBMS_Random.Value

Test Results
The embedded Excel file below summarises the results obtained over a grid of data points, with w in (1, 2, 3) and d in (8, 10, 12, 14, 16, 18).

The graphs tab below shows 3-d graphs of the number of rows processed and the CPU time for XFUN.

Notes

  • There is not much difference in performance between the two query versions, no doubt because the number of solution records is generally small compared with rows processed in the recursions
  • Notice that the timings correlate well with the rows processed, but not so well with the numbers of base records. The nature of the problem means that some of the randomised data sets turn out to be much harder to solve than others
  • Notice the estimated rows on step 36 of the execution plan for the pipelined function solution. The value of 8168 is a fixed value that Oracle assumes since it has no statistics to go on. We could improve this by using the (undocumented) cardinality hint to provide a smaller estimate
  • I extended my benchmarking framework for this article to report the intermediate numbers of rows processed, as well as the cardinality estimates and derived errors in these estimates (maximum for each plan). It is obvious from the nature of the problem that Oracle's Cost Based Optimiser (CBO) is not going to be able to make good cardinality estimates

Conclusions
Oracle's v11.2 implementation of the Ansii SQL feature recursive subquery factoring provides a means for solving the knapsack problem, in its multiple knapsack form, in SQL. The solution is not practical for large problems, for which procedural techniques that have been extensively researched should be considered. However, the techniques used may be of interest for combinatorial problems that are small enough to be handled in SQL, and for other types of problem in general.






A Simple SQL Solution for the Knapsack Problem (SKP-1)

A poster on OTN (Combination using pl/sql) recently asked for an SQL solution to a problem that turned out to be an example of the well known Knapsack Problem, for the case of a single knapsack. I posted an SQL query as a solution, and also a solution in PL/SQL because the SQL solution uses a feature only available in Oracle v11.2. In this article I explain how the solutions work and provide the results of a performance analysis that involved randomised test problems of varying computational difficulty. I have taken a more general form of problem than the original poster described, and the solutions here have been improved.

Update, 14 July 2013: I used the technique in response to another OTN post here, SQL for the Fantasy Football Knapsack Problem. I have extended the idea there to allow for fast approximate solutions making it viable for larger problems, and have also used a similar idea here, SQL for the Travelling Salesman Problem.

Knapsack Problem (1-Knapsack)
The various forms of knapsack problem have been studied extensively. The problems are known to be computationally difficult and many algorithms have been proposed for both exact and approximate solutions (see reference above). The SQL solution in this article is quite simple and will not be competitive in performance for larger problems in the form described here, but may be interesting for being implemented in pure SQL (and without using Oracle's Model clause, or a purely brute force approach). However, I have later extended the approach to allow for search limiting and have shown this to be viable for larger problems (see links at the top).

The problem can be stated informally, as follows: Given a set of items, each having positive weight and profit attributes, and a weight limit, find the combinations of items that maximise profit within the weight limit. Variant versions include the addition of multiple constraints (easy to handle), and inclusion of multiple knapsacks (more difficult). I also have a solution for the multiple knapsacks version described here (An SQL Solution for the Multiple Knapsack Problem (SKP-m)).

The difficulty of the problem arises from the number of possible combinations increasing exponentially with problem size. The number of these (not necessarily feasible) combinations, N(n,1), can be expressed in terms of the number of items, n, in two ways. First, we can use the well known binomial expression for the number of combinations of r items, summed from to :

N(n,1) =

where

Second, and more simply, we can observe that including an item in the combination, or not, is a binary choice, leading to:

N(n,1) =

This generalises easily to the expression for the multiple knapsack problem, with m knapsacks:

N(n,m) =

This can also be expressed using a binomial series as

N(n,m) =

Here, represents the number of combinations of r items from n, with being the number of assignments of the r items to m containers.

Let's look at a simple example problem having four items, with a weight limit of 9, as shown below:

Items

There are 24 possible combinations of these items, having from 0 to 4 items. These are depicted below:

Combinations

We can see that there are two optimal solutions in this case. How to find them using SQL?

SQL Solution

Oracle's v11.2 implementation of the Ansii standard Recursive Subquery Factoring can be used as the basis for an SQL solution. This would works as follows: Starting from each item in turn, add items recursively while remaining within the weight limit, and considering only items of id greater than the current id. The SQL looks like this, where a marker is added for leaf nodes, following an approach from the Amis technology blog:

WITH rsf (nxt_id, lev, tot_weight, tot_profit, path) AS (
SELECT id nxt_id, 0 lev, item_weight tot_weight, item_profit tot_profit, To_Char (id) path
  FROM items
 UNION ALL
SELECT n.id, 
       r.lev + 1, 
       r.tot_weight + n.item_weight,
       r.tot_profit + n.item_profit,
       r.path || ',' || To_Char (n.id)
  FROM rsf r
  JOIN items n
    ON n.id > r.nxt_id
   AND r.tot_weight + n.item_weight <= 9
) SEARCH DEPTH FIRST BY nxt_id SET line_no 
SELECT LPad (To_Char(nxt_id), lev + 1, '*') node,tot_weight, tot_profit,
       CASE WHEN lev >= Lead (lev, 1, lev) OVER (ORDER BY line_no) THEN 'Y' END is_leaf,
       path
  FROM rsf
 ORDER BY line_no

and the solution like this:

NODE       TOT_WEIGHT TOT_PROFIT I PATH
---------- ---------- ---------- - ------------------------------
1                   3         10   1
*2                  7         30 Y 1,2
*3                  8         40 Y 1,3
*4                  9         50 Y 1,4
2                   4         20   2
*3                  9         50 Y 2,3
3                   5         30 Y 3
4                   6         40 Y 4

8 rows selected.

The output contains 8 records, as opposed to the total of 23 non-null combinations, because only feasible items are joined, and permutations are avoided by the constraint that item ids increase along the path. Given positivity of weight and profit, we know that all solutions must be leaves, and we can represent the tree structure above in the following diagram:
Leaves
We can now use the recursive subquery factor as an input to a main query that selects one of the most profitable solutions, or alternatively to a further subquery factor that ranks the solutions in order of descending profit. In the latter case, the main query can select all the most profitable solutions.

In the solution I posted on the OTN thread, I included a subquery factor to restrict the final query section to leaf nodes only. This was because we know that the solutions must be leaf nodes, and usually it is more efficient to filter out non-solution records as early as possible. However, I later realised that the work involved in the filtering might outweigh the saving for the final section, and this turned out to be the case here, as shown in the performance analysis section below. Here are the two queries, without the leaf node filtering:

Query - KEEP

WITH rsf (id, lev, tot_weight, tot_profit, path) AS (
SELECT id, 0, item_weight, item_profit, To_Char (id)
  FROM items
 UNION ALL
SELECT n.id, 
       r.lev + 1, 
       r.tot_weight + n.item_weight,
       r.tot_profit + n.item_profit,
       r.path || ',' || To_Char (n.id)
  FROM rsf r
  JOIN items n
    ON n.id > r.id
   AND r.tot_weight + n.item_weight <= 100
)
SELECT Max (tot_weight) KEEP (DENSE_RANK LAST ORDER BY tot_profit) tot_weight,
       Max (tot_profit) KEEP (DENSE_RANK LAST ORDER BY tot_profit) tot_profit,
       Max (path) KEEP (DENSE_RANK LAST ORDER BY tot_profit) path,
       (Max (lev) KEEP (DENSE_RANK LAST ORDER BY tot_profit) + 1) n_items
  FROM rsf

Query - RANK

WITH rsf (id, lev, tot_weight, tot_profit, path) AS (
SELECT id, 0, item_weight, item_profit, To_Char (id)
  FROM items
 UNION ALL
SELECT n.id, 
       r.lev + 1, 
       r.tot_weight + n.item_weight,
       r.tot_profit + n.item_profit,
       r.path || ',' || To_Char (n.id)
  FROM rsf r
  JOIN items n
    ON n.id > r.id
   AND r.tot_weight + n.item_weight <= 100
)
, paths_ranked AS (
SELECT tot_weight, tot_profit, path, 
       Dense_Rank () OVER (ORDER BY tot_profit DESC) rnk_profit,
       lev
  FROM rsf
)
SELECT tot_weight tot_weight, 
       tot_profit tot_profit, 
       path path, 
       (lev + 1) n_items
  FROM paths_ranked
 WHERE rnk_profit = 1
 ORDER BY tot_weight DESC

Query Structure Diagram
QSD
It's worth noting that Oracle's proprietary recursive syntax, Connect By, cannot be used in this way because of the need to accumulate weights forward through the recursion. The new Ansii syntax is only available from v11.2 though, and I thought it might be interesting to implement a solution in PL/SQL that would work in earlier versions, following a similar algorithm, again with recursion.

PL/SQL Recursive Solution

This is a version in the form of a pipelined function, as I wanted to compare it with the SQL solutions, and be callable from SQL.
SQL

SELECT COLUMN_VALUE sol
  FROM TABLE (Packing_PLF.Best_Fits (100))
 ORDER BY COLUMN_VALUE

Package

CREATE OR REPLACE PACKAGE BODY Packing_PLF IS

PROCEDURE Write_Log (p_line VARCHAR2) IS
BEGIN
  NULL;
END Write_Log;

FUNCTION Best_Fits (p_weight_limit NUMBER) RETURN SYS.ODCIVarchar2List PIPELINED IS

  TYPE item_type IS RECORD (
                        item_id                 PLS_INTEGER,
                        item_index_parent       PLS_INTEGER,
                        weight_to_node          NUMBER);
  TYPE item_tree_type IS        TABLE OF item_type;
  g_solution_list               SYS.ODCINumberList;

  g_timer                       PLS_INTEGER := Timer_Set.Construct ('Pipelined Recursion');

  i                             PLS_INTEGER := 0;
  j                             PLS_INTEGER := 0;
  g_item_tree                   item_tree_type;
  g_item                        item_type;
  l_weight                      PLS_INTEGER;
  l_weight_new                  PLS_INTEGER;
  l_best_profit                 PLS_INTEGER := -1;
  l_sol                         VARCHAR2(4000);
  l_sol_cnt                     PLS_INTEGER := 0;

  FUNCTION Add_Node (  p_item_id               PLS_INTEGER,
                       p_item_index_parent     PLS_INTEGER, 
                       p_weight_to_node        NUMBER) RETURN PLS_INTEGER IS
  BEGIN

    g_item.item_id := p_item_id;
    g_item.item_index_parent := p_item_index_parent;
    g_item.weight_to_node := p_weight_to_node;
    IF g_item_tree IS NULL THEN

      g_item_tree := item_tree_type (g_item);

    ELSE

      g_item_tree.Extend;
      g_item_tree (g_item_tree.COUNT) := g_item;

    END IF;
    RETURN g_item_tree.COUNT;

  END Add_Node;

  PROCEDURE Do_One_Level (p_tree_index PLS_INTEGER, p_item_id PLS_INTEGER, p_tot_weight PLS_INTEGER, p_tot_profit PLS_INTEGER) IS

    CURSOR c_nxt IS
    SELECT id, item_weight, item_profit
      FROM items
     WHERE id > p_item_id
       AND item_weight + p_tot_weight <= p_weight_limit;
    l_is_leaf           BOOLEAN := TRUE;
    l_index_list        SYS.ODCINumberList;

  BEGIN

    FOR r_nxt IN c_nxt LOOP
      Timer_Set.Increment_Time (g_timer,  'Do_One_Level/r_nxt');

      l_is_leaf := FALSE;
      Do_One_Level (Add_Node (r_nxt.id, p_tree_index, r_nxt.item_weight + p_tot_weight), r_nxt.id, p_tot_weight + r_nxt.item_weight, p_tot_profit + r_nxt.item_profit);
      Timer_Set.Increment_Time (g_timer,  'Do_One_Level/Do_One_Level');

    END LOOP;

    IF l_is_leaf THEN

      IF p_tot_profit > l_best_profit THEN

        g_solution_list := SYS.ODCINumberList (p_tree_index);
        l_best_profit := p_tot_profit;

      ELSIF p_tot_profit = l_best_profit THEN

        g_solution_list.Extend;
        g_solution_list (g_solution_list.COUNT) := p_tree_index;

      END IF;

    END IF;
    Timer_Set.Increment_Time (g_timer,  'Do_One_Level/leaves');

  END Do_One_Level;

BEGIN

  FOR r_itm IN (SELECT id, item_weight, item_profit FROM items) LOOP

    Timer_Set.Increment_Time (g_timer,  'Root fetches');
    Do_One_Level (Add_Node (r_itm.id, 0, r_itm.item_weight), r_itm.id, r_itm.item_weight, r_itm.item_profit);

  END LOOP;

  FOR i IN 1..g_solution_list.COUNT LOOP

    j := g_solution_list(i);
    l_sol := NULL;
    l_weight := g_item_tree (j).weight_to_node;
    WHILE j != 0 LOOP

      l_sol := l_sol || g_item_tree (j).item_id || ', ';
      j :=  g_item_tree (j).item_index_parent;

    END LOOP;
    l_sol_cnt := l_sol_cnt + 1;
    PIPE ROW ('Solution ' || l_sol_cnt || ' (profit ' || l_best_profit || ', weight ' || l_weight || ') : ' || RTrim (l_sol, ', '));

  END LOOP;

  Timer_Set.Increment_Time (g_timer,  'Write output');
  Write_Log ('Profit ' || l_best_profit || ' has ' || l_sol_cnt || ' solutions...');
  Timer_Set.Write_Times (g_timer);

EXCEPTION
  WHEN OTHERS THEN
    Timer_Set.Write_Times (g_timer);
    RAISE;

END Best_Fits;

END Packing_PLF;

Performance Analysis

It will be interesting to see how the solution methods perform as problem size varies, and we will use my own performance benchmarking framework to do this. As the framework is designed to compare performance of SQL queries, I have converted the PL/SQL solution to operate as a pipelined function, and thus be callable from SQL, as noted above. I included a version of the SQL solution, with the leaf filtering mentioned above, XKPLV - this was based on XKEEP, with filtering as in the OTN thread.

Test Data Sets

Test data sets are generated as follows, in terms of two integer parameters, w and d:

Insert w items with sequential ids, and random weights and profits in the ranges 0-d and 0-1000, respectively, via Oracle's function DBMS_Random.Value. The maximum weight is fixed at 100.

Test Results

The embedded Excel file below summarises the results obtained over a grid of data points, with w in (12, 14, 16, 18, 20) and d in (16, 18, 20).

Notes

  • The two versions of the non-leaf SQL solution take pretty much the same time to execute, and are faster than the others
  • The leaf version of the SQL solution (XKPLV) is slower than the non-leaf versions, and becomes much worse in terms of elapsed time for the more difficult problems; the step-change in performance can be seen to be due to its greater memory usage, which spills to disk above a certain level
  • The pipelined function solution is significantly slower than the other solutions in terms of CPU time, and elapsed time, except in the case of the leaf SQL solution when that solution's memory usage spills to disk. The pipelined function continues to use more memory as the problem difficulty rises, until all available memory is consumed, when it throws an error (but this case is not included in the result set above)

Conclusions

Oracle's v11.2 implementation of the Ansii SQL feature recursive subquery factoring provides a simple solution for the knapsack problem, that cannot be achieved with Oracle's older Connect By syntax alone.

The method has been described here in its exact form that is viable only for small problems; however, I have later extended the approach to allow for search limiting and have shown this to be viable for larger problems.






Master-Detail Transaction Reconciliation in SQL (MDTM3)

This is the final article in a sequence of three on the subject of master-detail transaction matching. In the first article, Master-Detail Transaction Matching in SQL (MDTM1), I described the problem and divided it into two subproblems, the first being to identify all pairs of matching transactions and the second being to reconcile the pairs so that one transaction matches against at most one other transaction. The underlying motivation here comes from the problem of reconciling intra-company credit and debit transactions where fields may need to match directly, or may need to match after some mapping function is applied, including inversion (contra-matching). We have taken a simple prototype problem defined on Oracle's system tables where only matching conditions are specified. It should be straightforward to extend the techniques demonstrated to more general matching conditions (I have done so myself on the real business problem that prompted this analysis).

The first article developed a series of queries to solve the first subproblem using the idea of pre-aggregation of the detail records as a key performance-enhancing feature. This resulted in a best query that used a temporary table and achieved a time variation that was quadratic in the number of master transactions (we kept the numbers of details per master fixed).

The second article, Holographic Set Matching in SQL (MDTM2), took the aggregation a step further, using list aggregation to bypass direct detail set matching altogether, and this enabled linear time variation.

In this third article, we take the last, linear-time query and extend it to solve the second subproblem, providing a solution to the overall problem in a single query that shows the same linear-time variation property in our test results. The sample problem will be the same as in the previous article.

Output Specification
The output will be ordered first by section, then by group, then by transaction unique identifier, with paired records appearing together using the first transaction for ordering within the group. The sections are defined thus:

  • Reconciled pairs - two records for each matching pair, with no transaction appearing in more than one pair
  • Matched but unreconciled transactions - transactions that match others but could not be reconciled because their matching partners are all paired off against other transactions
  • Unmatched transactions - transactions not matching any other transaction

Queries
We'll include the best query from the last article (L2_SQF), as well as the new query (RECON) that extends it to solve the overall problem.

  • L2_SQF - solves first subproblem by list aggregation without direct detil matching
  • RECON - extends L2_SQF to solve the overall problem using a sequence of query subfactors

The first query will not be described below, as it appeared in the previous article but will be included in the results section for comparison purposes.

Query Structure Diagram (QSD)

Query Text

WITH rns AS (
SELECT r_owner,
       r_constraint_name,
       Row_Number () OVER (ORDER BY r_owner, r_constraint_name) - 1 rn
  FROM con_cp
 WHERE constraint_type	    = 'R'
 GROUP BY 
       r_owner,
       r_constraint_name
), rch AS ( 
SELECT r_owner,
       r_constraint_name,
       Chr (Floor (rn / 128)) ||
       Chr ((rn - 128 * Floor (rn / 128))) chr_rank
  FROM rns
), tab AS (
SELECT t.owner,
       t.table_name,
       t.ROWID                   row_id,
       Count(c.ROWID)            n_det,
       Listagg (r.chr_rank, '') WITHIN GROUP (ORDER BY r.chr_rank) lagg
  FROM tab_cp                    t
  JOIN con_cp                    c
    ON c.owner                   = t.owner
   AND c.table_name              = t.table_name
   AND c.constraint_type         = 'R'
  JOIN rch                       r
    ON r.r_owner                 = c.r_owner
   AND r.r_constraint_name       = c.r_constraint_name
 GROUP BY
        t.owner,
        t.table_name,
        t.ROWID
), dup as (
SELECT t1.owner                  owner_1,
       t1.table_name             table_name_1,
       t2.owner                  owner_2,
       t2.table_name             table_name_2,
       t1.n_det                  n_det
  FROM tab                       t1
  JOIN tab                       t2
    ON t2.lagg                   = t1.lagg
   AND t2.row_id                 > t1.row_id
), btw AS (
SELECT owner_1       owner_1_0,
       table_name_1  table_name_1_0,
       owner_1,
       table_name_1,
       owner_2,
       table_name_2,
       n_det
  FROM dup
 UNION
SELECT owner_1,
       table_name_1,
       owner_2,
       table_name_2,
       owner_1,
       table_name_1,
       n_det
  FROM dup
), grp AS (
SELECT owner_1_0,
       table_name_1_0,
       owner_1,
       table_name_1,
       owner_2,
       table_name_2,
       n_det,
       Least (owner_1 || '/' || table_name_1, Min (owner_2 || '/' || table_name_2)
         OVER (PARTITION BY owner_1, table_name_1)) grp_uid
  FROM btw
), rnk AS (
SELECT owner_1_0,
       table_name_1_0,
       owner_1,
       table_name_1,
       Dense_Rank () OVER  (PARTITION BY grp_uid ORDER BY owner_1, table_name_1) r1,
       owner_2,
       table_name_2,
       Dense_Rank () OVER  (PARTITION BY grp_uid ORDER BY owner_2, table_name_2) r2,
       n_det,
       grp_uid
  FROM grp
), rec AS (
SELECT owner_1_0,
       table_name_1_0,
       owner_1,
       table_name_1,
       owner_2,
       table_name_2,
       n_det,
       grp_uid
  FROM rnk
 WHERE (r2 = r1 + 1 AND Mod (r1, 2) = 1) OR (r1 = r2 + 1 AND Mod (r2, 2) = 1)
), rcu AS (
SELECT owner_1_0,
       table_name_1_0,
       owner_1,
       table_name_1,
       owner_2,
       table_name_2,
       n_det,
       grp_uid
  FROM rec
 UNION  
SELECT owner_1,
       table_name_1,
       owner_1,
       table_name_1,
       NULL,
       NULL,
       n_det,
       grp_uid
  FROM grp
 WHERE (owner_1, table_name_1) NOT IN (SELECT owner_1, table_name_1 FROM rec)
 UNION  
SELECT owner,
       table_name,
       owner,
       table_name,
       NULL,
       NULL,
       n_det,
       NULL
  FROM tab
 WHERE (owner, table_name) NOT IN (SELECT owner_1, table_name_1 FROM btw)
)
SELECT
       owner_1,
       table_name_1,
       owner_2,
       table_name_2,
       n_det,
       grp_uid
  FROM rcu
 ORDER BY grp_uid,
       CASE WHEN grp_uid IS NULL THEN 3 
               WHEN owner_2 IS NULL THEN 2
               ELSE 1
          END,
       owner_1_0,
       table_name_1_0,
       owner_1,
       table_name_1

How it Works
The query proceeds by ten stages using query subfactors. The first three stages correspond to query L2_SQF which then has a main query, whereas we now have another six stages before the main query, as shown:

  1. Rank the distinct details [Group by matching fields, then use Row_Number to rank by same]
  2. Convert ranks to base 128 [Use Floor() and Chr() functions; uses 2 characters here]
  3. Aggregate detail ranks for master [Use Listagg on the rank fixed-length string]
  4. Get all matching pairs one-way [Self-join on matching aggregate and second rowid greater]
  5. Union in the reverse pairs, with sorting column [Add in records with uids swapped, but keep uid 1 separately for sorting]
  6. Assign grouping field to each pair [Take minimum of uid 2 over uid 1, or uid 1 if smaller]
  7. Rank each side of pair within its group [Use Dense_Rank() over grouping, ordering by uids]
  8. Retain odd-even sequentially ranked pairs [Retain pairs with ranks (1,2) or (3,4) etc. and the reverses]
  9. Add in unmatched and matched but unreconciled [3-way union: first the reconciled; then the matched but unreconciled; then unmatched]
  10. Sort by the source uid 1, then the current uid 1 [Sort key ensures that reconciled pairs stay together within their matching group]

Notes:

  • In our matching-only sample problem, matching transactions form mutually matching sets, whereas for contra-matching, there are pairs of contra-matching sets as discussed in the first article. The grouping subqueries will therefore differ in the latter case, and for example, pairing could be by matching numerical rank within the respective sets
  • The final subquery factor could be incorporated in the main query, but I retain it because the benchmarking framework does not support unions in the main query, and CBO optimises it away in any case

Results
Both queries were run on the same sample problem as in the previous article. The output comparison gives the output listings for width parameter of 1, which corresponds to the tables and constraints on my v11.2 system copied to the test tables with owner prefix '_0' added. The timings and statistics are for widths from 1 to 6.

Output Comparison

The output file has tabs for the output from both queries, and a tab with examples from each of the three sections for the second. Notice that OEHR_EMPLOYEES and OEHR_JOB_HISTORY form a reconciled pair, with three detail records, while EMPLOYEES and JOB_HISTORY are unmatched, with four detail records. This is because, as I mentioned in the first article, I have added an extra record to each of the latter tables' detail tables (i.e. foreign key constraints), the extra record being a duplicate of one of the other three (in terms of the matching fields), but a different duplicate in each case. This tests correct handling of duplicates within the detail sets.

Performance Comparison
Click on the query name in the file below to jump to the execution plan for the largest data point, and click the tabs to see different views on the performance obtained.

  • The timings in the file above are roughly consistent with linear-time variation with problem size; if anything L2_SQF appears sublinear, but the times are fairly small and there was some other activity on the PC at the time
  • At the largest data point, RECON takes 5 times as much CPU time as L2_SQF, and 9 times as much elapsed time
  • The differences between elapsed and CPU times for RECON are indicative of significant file I/O activity. This shows up in the disk reads and writes summaries on the statistics tab, and in more detail in the Plans tab, and is caused mainly by reading and writing of the subquery factors to and from disk
  • The other main significant factor in the increase in times for the RECON query is the additional sorting; see, for example, steps 31 and 32 in the plan. These effects are the result of the additional subquery factors that were needed to achieve the final result

Conclusions

  • This three-part sequence of articles has focussed on a special category of problem within SQL, but has highlighted a range of SQL techniques that are useful generally, including:
    • Subquery factors
    • Temporary tables
    • Analytic functions
    • Set matching by list aggregation
    • Compact storage of unique identifiers by ranking and base-conversion via the Chr() function
  • We have also noted different ways of matching sets, including use of the MINUS set operator and the NOT EXISTS construct, and discussed ways of handling issues such as duplication within a set, and directionality of the set operators
  • The importance of polynomial order of solution performance for efficiency has been illustrated dramatically
  • The final SQL solution provides a good illustration of the power of modern SQL to solve complex problems using set-based logic combined with sequence in a simpler and faster way than the more conventional procedural approach
  • The subquery-sequence approach to SQL is well suited to diagrammatic design techniques
  • It is often better to solve complex real-world problems by first working with simpler prototypes






Holographic Set Matching in SQL (MDTM2)

This article is the second in a sequence of three dealing with a very general class of problems in SQL, and exploring various techniques to find efficient solutions. In the first article, Master-Detail Transaction Matching in SQL (MDTM1), the problem was outlined and divided into two subproblems, of which the first was solved in several variant SQL statements with performance analysis. This second article, takes the most efficient method and applies two new techniques to further improve performance. The third article, Master-Detail Transaction Reconciliation in SQL (MDTM3), adds a sequence of subquery factors to the best solution for the first subproblem to achieve an efficient solution to the overall problem within a single SQL statement.

The holographic principle is a mathematical principle that the total information contained in a volume of space corresponds to an equal amount of information contained on the boundary of that space. - Holographic Principle.

In my last article, I made large performance gains in SQL queries matching sets of detail records by obtaining aggregates of the sets in a subquery factor and matching those at master level before matching the detail sets directly. The performance gain came from the fact that the aggregation is cheap compared to matching sets of records and allows many matching pair candidates to be discarded before doing the expensive direct set matching. However, all of the actually matching transactions would have been directly matched and probably more, and it occurred to me to wonder whether it might not be possible to use aggregate matching to replace detail set matching altogether.

This article develops the previous one by taking the same sample transaction matching problem and adding queries that use the Oracle 11.2 function Listagg to allow just this replacement. This is possible so long as the list-aggregated detail matching fields do not exceed 4,000 characters. If that were to happen then some other aggregation technique would be needed, perhaps a user-defined CLOB version of Listagg. However, it's possible to extend the range of applicability by aggregating identifiers smaller than the actual fields, as I'll discuss at the end.

We'll keep the fastest query from the previous article, and add three new queries:

Query Variations

  • MIN_NE - Detail grouping in temporary table, with set matching (GTT_NE previously)
  • LAG_SQF - Detail grouping by Listagg only in subquery factor
  • LAG_NE - Detail grouping by Listagg in subquery factor, with set matching
  • LAG_GTT - Detail grouping by Listagg only in temporary table
************
MIN_NE
************
  WITH tab AS (
SELECT t.owner,
       t.table_name,
       t.ROWID                   row_id,
       Sum (c.n_con)             n_det,
       Min (c.r_constraint_name) min_det,
       Max (c.r_constraint_name) max_det
  FROM tab_cp                    t
  JOIN grp_gtt                   c
    ON c.owner                   = t.owner
   AND c.table_name              = t.table_name
 GROUP BY
        t.owner,
        t.table_name,
        t.ROWID
)
SELECT
       t1.owner                  owner_1,
       t1.table_name             table_name_1,
       t2.owner                  owner_2,
       t2.table_name             table_name_2,
       t1.n_det                  n_det
  FROM tab                       t1
  JOIN tab                       t2
    ON t2.n_det                  = t1.n_det
   AND t2.min_det                = t1.min_det
   AND t2.max_det                = t1.max_det
   AND t2.row_id                 > t1.row_id
 WHERE NOT EXISTS (
SELECT 1
  FROM grp_gtt                   d1
 WHERE d1.owner                  = t1.owner
   AND d1.table_name             = t1.table_name
   AND (
   NOT EXISTS (
SELECT 1
  FROM grp_gtt                   d2
 WHERE d2.owner                  = t2.owner
   AND d2.table_name             = t2.table_name
   AND d2.r_owner                = d1.r_owner
   AND d2.r_constraint_name      = d1.r_constraint_name
   AND d2.n_con                  = d1.n_con
)))
 ORDER BY t1.owner,
       t1.table_name,
       t2.owner,
       t2.table_name

************
LAG_SQF
************
  WITH tab AS (
SELECT t.owner,
       t.table_name,
       t.ROWID                   row_id,
       Count(c.ROWID)            n_det,
       Listagg (c.r_owner||c.r_constraint_name, '') WITHIN GROUP (ORDER BY c.r_owner||c.r_constraint_name) lagg
  FROM tab_cp                    t
  JOIN con_cp                    c
    ON c.owner                   = t.owner
   AND c.table_name              = t.table_name
   AND c.constraint_type         = 'R'
 GROUP BY
        t.owner,
        t.table_name,
        t.ROWID
)
SELECT
       t1.owner                  owner_1,
       t1.table_name             table_name_1,
       t2.owner                  owner_2,
       t2.table_name             table_name_2,
       t1.n_det                  n_det
  FROM tab                       t1
  JOIN tab                       t2
    ON t2.n_det                  = t1.n_det
   AND t2.lagg                   = t1.lagg
   AND t2.row_id                 > t1.row_id
 ORDER BY t1.owner,
       t1.table_name,
       t2.owner,
       t2.table_name

************
LAG_NE
************
  WITH tab AS (
SELECT t.owner,
       t.table_name,
       t.ROWID                   row_id,
       Sum (c.n_con)             n_det,
       Listagg (c.r_owner||c.r_constraint_name, '') WITHIN GROUP (ORDER BY c.r_owner||c.r_constraint_name) lagg
  FROM tab_cp                    t
  JOIN grp_gtt                   c
    ON c.owner                   = t.owner
   AND c.table_name              = t.table_name
 GROUP BY
        t.owner,
        t.table_name,
        t.ROWID
)
SELECT
       t1.owner                  owner_1,
       t1.table_name             table_name_1,
       t2.owner                  owner_2,
       t2.table_name             table_name_2,
       t1.n_det                  n_det
  FROM tab                       t1
  JOIN tab                       t2
    ON t2.n_det                  = t1.n_det
   AND t2.lagg                   = t1.lagg
   AND t2.row_id                 > t1.row_id
 WHERE NOT EXISTS (
SELECT 1
  FROM grp_gtt                   d1
 WHERE d1.owner                  = t1.owner
   AND d1.table_name             = t1.table_name
   AND (
   NOT EXISTS (
SELECT 1
  FROM grp_gtt                   d2
 WHERE d2.owner                  = t2.owner
   AND d2.table_name             = t2.table_name
   AND d2.r_owner                = d1.r_owner
   AND d2.r_constraint_name      = d1.r_constraint_name
   AND d2.n_con                  = d1.n_con
)))
 ORDER BY t1.owner,
       t1.table_name,
       t2.owner,
       t2.table_name

************
LAG_GTT
************
SELECT
       t1.owner                  owner_1,
       t1.table_name             table_name_1,
       t2.owner                  owner_2,
       t2.table_name             table_name_2,
       t1.n_det                  n_det
  FROM tab_gtt                   t1
  JOIN tab_gtt                   t2
    ON t2.n_det                  = t1.n_det
   AND t2.lagg                   = t1.lagg
   AND t2.row_id                 > t1.row_id
 ORDER BY t1.owner,
       t1.table_name,
       t2.owner,
       t2.table_name

The two temporary tables are as follows, with * marking unique keys:
tab_gtt

  • owner*
  • table_name*
  • row_id
  • lagg
  • n_det

A unique index is defined on tab_gtt:
tab_gtt_uk

  • owner
  • table_name

A non-unique index is defined on tab_gtt:
tab_gtt_N1

  • lagg

grp_gtt

  • owner*
  • constraint_name*
  • r_owner*
  • r_constraint_name*
  • n_con

A unique index is defined on grp_gtt:
grp_gtt_uk

  • owner
  • constraint_name
  • r_owner
  • r_constraint_name

Performance Analysis

We presented four query variations above, and in this section give the results of benchmarking these queries across a 1-dimensional data domain obtained by copying the system views once, then by successive powers of two up to 32 times into my test tables described in the previous article. The problem sizes are as follows:
Record Counts

Timings and Statistics
Click on the query name in the file below to jump to the execution plan for the largest data point.

Comparison

The figures above are for the largest data pont, W32. The following points can be made:

  • The query that performed best in the earlier article is now worst compared with the new queries
  • The best query uses Listagg within a subquery factor and is more than 200 times faster than the worst at the largest data point
  • Moving the new listagg-based subquery factor into a temporary table worsens performance because the index is not used
  • The statistics tab shows that performance variation is now linear with problem size for the new Listagg queries, which is why they inevitably outperform the old one at large enough problem size

Subquery Factors and Temporary Tables
Replacing a subquery factor by a temporary table can only help if indexed accesses are beneficial.

Execution Plan Hash Values
In my last article, I noted that the plan hash values differed for all queries between data points, although the plans were essentially the same, and surmised that this was due to the subquery factor internal naming system. LAG_GTT is the only query here that makes no use of subquery factors, and this is the only one that retains the same plan hash value, thus bearing out the surmise.

Extending Listagg Applicability

If there are a large number of matching fields then the Listagg limit of 4,000 characters could be hit in quite a small number of details for a master. It's not difficult to write a CLOB version of Listagg, but one way of mitigating the restriction would be to aggregate not the actual matching fields, but the ranking of the set within all distinct detail sets. A further reduction in the size of the aggregated values can be obtained by storing the ranking in a high number-base, rather than base 10, as a zero-left-padded string. If the database character set is UTF8 (as is my 11.2 XE database), base 128 is possible, while extended Ascii character sets should allow base 256. The number of characters assigned to the ranking value determines how many distinct sets and how many detail records per master record are allowed with the standard Listagg function, according to the table below (for UTF8):

Chars  Distinct Sets  Details/Master
=====  =============  ==============
    1            128           4,000
    2         16,384           2,000
    3      2,097,152           1,333
    4    268,435,456           1,000

New Query L2_SQF

WITH rns AS (
SELECT r_owner,
       r_constraint_name,
       Row_Number () OVER (ORDER BY r_owner, r_constraint_name) - 1 rn
  FROM con_cp
 WHERE constraint_type	    = 'R'
 GROUP BY 
       r_owner,
       r_constraint_name
), rch AS ( 
SELECT r_owner,
       r_constraint_name,
       Chr (Floor (rn / 128)) ||
       Chr ((rn - 128 * Floor (rn / 128))) chr_rank
  FROM rns
), tab AS (
SELECT t.owner,
       t.table_name,
       t.ROWID                   row_id,
       Count(c.ROWID)            n_det,
       Listagg (r.chr_rank, '') WITHIN GROUP (ORDER BY r.chr_rank) lagg
  FROM tab_cp                    t
  JOIN con_cp                    c
    ON c.owner                   = t.owner
   AND c.table_name              = t.table_name
   AND c.constraint_type         = 'R'
  JOIN rch                       r
    ON r.r_owner                 = c.r_owner
   AND r.r_constraint_name       = c.r_constraint_name
 GROUP BY
        t.owner,
        t.table_name,
        t.ROWID
)
SELECT t1.owner                  owner_1,
       t1.table_name             table_name_1,
       t2.owner                  owner_2,
       t2.table_name             table_name_2,
       t1.n_det                  n_det
  FROM tab                       t1
  JOIN tab                       t2
    ON t2.lagg                   = t1.lagg
   AND t2.row_id                 > t1.row_id
 ORDER BY t1.owner,
       t1.table_name,
       t2.owner,
       t2.table_name

QSDs

Record Counts
The same set of data points has been used for the new query (L2_SQF) with the best of the earlier ones for comparison (LAG_SQF). The record counts have slightly increased. Note that the constraints per table is for all constraints, not just foreign keys.

Timings and Statistics
Click on the query name in the file below to jump to the execution plan for the largest data point.

Comparison

The figures above are for the largest data pont, W32. The following points can be made:

  • The new query is about 20% slower than the old one
  • Performance variation remains linear with problem size for the new query
  • The test problem has very few details per master, but the relative performances may change for real problems

Conclusions

This article has used a new idea that I termed holographic set matching to improve performance relative to the queries in my previous article on Master-Detail Transaction Matching in SQL (MDTM1), achieving linear time variation with size, compared with the earlier quadratic time. Although the new Oracle 11.2 function Listagg has been used, the method can by applied in earlier versions by adding a user-defined list aggregation function, which is easy to do. The third article in this sequence, Master-Detail Transaction Reconciliation in SQL (MDTM3), solves the overall problem described in the first article by adding a sequence of subquery factors to the query developed above.






Master-Detail Transaction Matching in SQL (MDTM1)

This article is the first in a sequence of three dealing with a very general class of problems in SQL, and exploring various techniques to find efficient solutions. In this first article, the problem is outlined and is divided into two subproblems, of which the first is solved here in several variant SQL statements with performance analysis. The second article, Holographic Set Matching in SQL, takes the most efficient method and applies two new techniques to further improve performance. The third article, Master-Detail Transaction Reconciliation in SQL (MDTM3), adds a sequence of subquery factors to the best solution for the first subproblem to achieve an efficient solution to the overall problem within a single SQL statement.

The General Problem

We consider a transaction with a two-level structure consisting of a header (or master) and lines (or details) linked to the header, and the problem is to pair off transactions by matching, or contra-matching, subsets of the fields at both header and line level. This kind of problem can arise in the context of reconciliation of debit and credit transactions where both transactions are in the same system but are entered separately by two different groups and have no explicit linkage. Typically, in order for the transactions to match at header level, several fields such as transaction class have to be equal, while others such as credit and debit amounts have to be inverse, while others again such as unique identifiers will not match. At the line level, the same applies and matched lines also need to pair off against each other for the transaction to be considered a match. The first part of the problem is to identify all matching (or contra-matching) pairs, and this article will focus on that, while the second (and optional) part, that of pairing off, will be the subject of a later article.

To see why performance might be an important issue for this type of problem, consider the number of possible comparisons for an example with 10,000 headers each having 100 lines. In this case there would be 100,000,000 pairs of transactions, counting both ways, and 50,000,000 counting one way. A similar calculation gives 10,000 (not 5,000!) line comparisons per header-pair, and hence 500,000,000,000 line comparisons in total. The key of course is to minimise the number of comparisons made explicitly.

We will solve the problem through a single SQL query which will be developed through several versions, using a test example problem based on Oracle standard tables. The queries will be tested using my own SQL benchmarking framework, mentioned in earlier articles, and performance characteristics analysed. This will illustrate some performance aspects of the use of subquery factors and temporary tables, among other things.

Matching and Contra-Matching Sets

In the ERD above, each transaction falls into a logical Match Status Set, where the sets are of four distinct types:

  • Unmatched - a single set for transactions having no matching or contra-matching transaction
  • Matched - a set for each group of mutually matching transactions
  • Contra-Matched A -?a set for each group of transactions that all contra-match to a corresponding B-set
  • Contra-Matched B -?a set for each group of transactions that all contra-match to a corresponding A-set

We may define our problem without contra-matching fields, in which case only the first two types of set will be present; we may also have the case where only contra-matching is possible (likely the most common); and a special case may arise where both matching and contra-matching fields are present but where all contra-matching fields may have self-inverse values (for example amounts of zero) and those records having only self-inverse values might be best regarded as falling into one of the first two types of set.

The Sample Problem - Tables and Foreign Key Constraints

We will use two of the Oracle database system views as the basis for our sample problem. The master entity will be the Oracle table defined in the view all_tables, and the detail entity will be the foreign key constraint contained as a subentity in the view all_constraints. The views themselves are very complicated and it is better for our purposes to copy their records into new tables, and for performance testing we'll copy them multiple times according to the value of a dimensional parameter, using the parameter as a suffix on the owner and table name fields. The sample problem will involve matching only, and tables are defined to match if they have the same set of foreign key references, where the references are defined by the referenced owners and constraint names. As tables without foreign keys all match trivially, we'll filter these out in the queries.

The table and constraint entities can be represented by the following ERD:

The tables are, with * marking primary keys:
tab_cp

  • owner*
  • table_name*
  • description

con_cp

  • owner*
  • constraint_name*
  • table_name
  • constraint_type
  • r_owner
  • r_constraint_name
  • description

Indexes are defined on the two foreign keys on con_cp:
con_tab_fk_N1

  • owner
  • table_name

con_con_fk_N2

  • r_owner
  • r_constraint_name

The embedded Excel file below gives the solution for my 11g XE database, for the first problem, of identifying all matches.

Solution Methods
This problem might be considered to divide into two subproblems. The first is to identify all the matching pairs, while the second is to take those matching pairs and eliminate duplicate instances, so that each master record matches against at most one other record. This may reduce the number of master records that have matches; for example, if a matching set has three master records, then only two of them will be matched, against each other, in the final solution. We will consider the first subproblem in this article and the second in a later article.

To find the solution to the first subproblem in SQL, the obvious approach is simply to join the master table to itself to form the set of possible matching pairs, then to apply criteria to filter out any pairs that don't match. Obviously, we can immediately apply a constraint to avoid selecting the same pair twice by requiring that the rowid of the first record be higher than that of the second. This will halve the number of pairs considered, reducing the initial set of pairs from n! to n!/2 (where ! denotes the mathematical factorial function), and also halving the number after applying any other conditions.

Matching Detail Sets with MINUS Operator
The master-level criteria may be easy enough to apply, using conditions in the join clause, but the detail criteria are more difficult because we have to match two sets of records for any given pair of master records. This leads us to think of Oracle's set operators, specifically the MINUS operator that subtracts one set from another. Consider the matching pair on line 4028 of the Excel file above, with he solution for our example problem. This shows a match between the two tables OEHR_EMPLOYEES and OEHR_JOB_HISTORY in the TWODAYPLUS_0 schema, each of which has three foreign keys. The three constraints on each of these tables reference the same keys in the same schema, namely DEPT_ID_PK, JOB_ID_PK, EMP_EMP_ID_PK. The following query returns no records:

SELECT r_owner,
       r_constraint_name
  FROM con_cp
 WHERE constraint_type = 'R'
   AND table_name = 'OEHR_EMPLOYEES'
   AND owner = 'TWODAYPLUS_0'
 MINUS
SELECT r_owner,
       r_constraint_name
  FROM con_cp
 WHERE constraint_type = 'R'
   AND table_name = 'OEHR_JOB_HISTORY'
   AND owner = 'TWODAYPLUS_0'

Perhaps then the detail set matching could be effected by a NOT EXISTS clause on the above query with the hard-coded owner and table_name replaced by correlation columns from the main query? There are two problems with this arising from the way Oracle's set operators work. First, if there were any extra foreign keys in the second table the query would still return no records, as it returns only records that are in the first query section and not in the second, thus showing a false match. Second, Oracle views a set in this context as being the set of distinct records, so if some records are duplicated in either table, but differently from the other one then again a false match is shown. These two tables also exist in Oracle's HR demo schema, without the OEHR_ prefix. In order to show the problem I added an extra field in each table with a foreign key matching one already present, as follows:

  • EMPLOYEES.REL_EMP_ID -> EMP_EMP_ID_PK
  • JOB_HISTORY.REL_JOB_ID -> JOB_ID_PK

Now the query above with new schema and table names still returns no records although in our terms the detail record sets are different: EMPLOYEES has set (DEPT_ID_PK, JOB_ID_PK, EMP_EMP_ID_PK, EMP_EMP_ID_PK), while JOB_HISTORY has set (DEPT_ID_PK, JOB_ID_PK, JOB_ID_PK, EMP_EMP_ID_PK). The solution to this problem is of course that we need to group the detail records by the matching fields and add a count, as follows, using our copied schema HR_0:

SELECT r_owner,
       r_constraint_name,
       Count(*)
  FROM con_cp
 WHERE constraint_type = 'R'
   AND table_name = 'EMPLOYEES'
   AND owner = 'HR_0'
 GROUP BY r_owner,
       r_constraint_name
 MINUS
SELECT r_owner,
       r_constraint_name,
       Count(*)
  FROM con_cp
 WHERE constraint_type = 'R'
   AND table_name = 'JOB_HISTORY'
   AND owner = 'HR_0'
 GROUP BY r_owner,
       r_constraint_name

This returns two records:

R_OWNER  R_CONSTRAINT_NAME    COUNT(*)
=======  =================    ========
HR_0     EMP_EMP_ID_PK        2
HR_0     JOB_ID_PK            1

As for the first problem, this can be solved in two ways, either by repeating the NOT EXISTS clause with the two sections reversed, or by ensuring separately that the two record sets have the same numbers of records - if they don't they can't match, and if they do then the MINUS operator works. Obviously the first solution is going to double the work involved, while the second incurs a cost associated with the counting process but that's offset by avoidance of the NOT EXISTS execution.

Matching Detail Sets with nested NOT EXISTS Operator
If we consider the MINUS query above before we added grouping, it seems likely that Oracle would evaluate the outer NOT EXISTS by obtaining both record sets, then applying the MINUS opersator, before checking that no records are returned. This would seem inefficient since the outer condition fails if any single record is in the first set but not in the second, so one would want to truncate processing on finding a first such record. This suggests an alternative that might be more effficient, that uses another NOT EXISTS nested within the outer one, which would apply to the following subquery:

SELECT 1
  FROM con_cp c1
 WHERE c1.constraint_type = 'R'
   AND c1.table_name = 'OEHR_EMPLOYEES'
   AND c1.owner = 'TWODAYPLUS_0'
   AND NOT EXISTS (
SELECT 1
  FROM con_cp c2
 WHERE c2.constraint_type = 'R'
   AND c2.table_name = 'OEHR_JOB_HISTORY'
   AND c2.owner = 'TWODAYPLUS_0'
   AND c2.r_owner = c1.r_owner
   AND c2.r_constraint_name = c1.r_constraint_name
)

Here we have not included the grouping solution because it is complicated within this structure, but if the detail table were replaced by either a subquery factor or a temporary table where the grouping were already done, then (as we'll see) this would work just by adding in an equality condition on the count fields. Again, if we know that the record counts are the same the reverse clause is unnecessary.

Pre-Matching Detail Sets by Aggregates
We noted above that the detail sets can only match if they have the same numbers of records, and that this could be used to avoid doing the set matching twice in opposite orders. We also noted that the work done in counting would be offset by the avoidance of expensive set matching for those pairs that don't have matching counts. In fact, we can extend this idea to all possible aggregates on the detail record set matching fields, and this will likely result in fewer set matchings in the overall query execution. In our simple test problem we will add minimum and maximum aggregates on the r_constraint_name field, giving the following join conditions, prior to the set matching clause, and where tab represents a subquery factor that computes the aggregates:

  FROM tab                      t1
  JOIN tab                      t2
    ON t2.n_det                 = t1.n_det
   AND t2.min_det               = t1.min_det
   AND t2.max_det               = t1.max_det
   AND t2.row_id                > t1.row_id

Subquery Factors and Temporary Tables
Owing to the importance of aggregation at table level, as explained in the last section above, all query variations considered will include a subquery factor, tab, that does this aggregation. However, we have also noted the need to group and count at the level of detail records, and as this grouped record set needs to be used twice, for each member of a potential matching master pair, it would also seem an obvious candidate for a subquery factor. When we try this though, we'll see that the query structure now precludes the use of indexes within the detail matching subquery and so we'll also implement a query that uses a temporary table where the grouping and counting is done in advance.

Query Variations
We will test five query variations, as shown below, where MI and NE denote, respectively, the MINUS and NOT EXISTS methods of detail set matching.

  • INL_MI - Detail grouping directly
  • SQF_NE - Detail grouping in subquery factor
  • GTT_NE - Detail grouping in temporary table
  • GTT_NE_X - As GRP_GTT_NE but table-level count aggregation only
  • GTT_MI - As GRP_GTT_NE but with MINUS
************
INL_MI
************
  WITH tab AS (
SELECT t.owner,
       t.table_name,
       t.ROWID                   row_id,
       Count(c.ROWID)            n_det,
       Min (c.r_constraint_name) min_det,
       Max (c.r_constraint_name) max_det
  FROM tab_cp                    t
  JOIN con_cp                    c
    ON c.owner                   = t.owner
   AND c.table_name              = t.table_name
   AND c.constraint_type         = 'R'
 GROUP BY
        t.owner,
        t.table_name,
        t.ROWID
)
SELECT
       t1.owner                  owner_1,
       t1.table_name             table_name_1,
       t2.owner                  owner_2,
       t2.table_name             table_name_2,
       t1.n_det                  n_det
  FROM tab                       t1
  JOIN tab                       t2
    ON t2.n_det                  = t1.n_det
   AND t2.min_det                = t1.min_det
   AND t2.max_det                = t1.max_det
   AND t2.row_id                 > t1.row_id
 WHERE NOT EXISTS (
SELECT c2.r_owner,
       c2.r_constraint_name,
       Count(*)
  FROM con_cp                    c2
 WHERE c2.owner                  = t2.owner
   AND c2.table_name             = t2.table_name
   AND c2.constraint_type        = 'R'
 GROUP BY c2.r_owner,
       c2.r_constraint_name
MINUS
SELECT c1.r_owner,
       c1.r_constraint_name,
       Count(*)
  FROM con_cp                    c1
 WHERE c1.owner                  = t1.owner
   AND c1.table_name             = t1.table_name
   AND c1.constraint_type        = 'R'
 GROUP BY c1.r_owner,
       c1.r_constraint_name
)
 ORDER BY t1.owner,
       t1.table_name,
       t2.owner,
       t2.table_name

************
SQF_NE
************
  WITH det AS (
SELECT owner,
       table_name,
       r_owner,
       r_constraint_name,
       Count(*)                  n_dup
  FROM con_cp
 WHERE constraint_type           = 'R'
 GROUP BY owner,
       table_name,
       r_owner,
       r_constraint_name
), tab AS (
SELECT t.owner,
       t.table_name,
       t.ROWID                   row_id,
       Sum (c.n_dup)             n_det,
       Min (c.r_constraint_name) min_det,
       Max (c.r_constraint_name) max_det
  FROM tab_cp                    t
  JOIN det                       c
    ON c.owner                   = t.owner
   AND c.table_name              = t.table_name
 GROUP BY
        t.owner,
        t.table_name,
        t.ROWID
)
SELECT
       t1.owner                  owner_1,
       t1.table_name             table_name_1,
       t2.owner                  owner_2,
       t2.table_name             table_name_2,
       t1.n_det                  n_det
  FROM tab                       t1
  JOIN tab                       t2
    ON t2.n_det                  = t1.n_det
   AND t2.min_det                = t1.min_det
   AND t2.max_det                = t1.max_det
   AND t2.row_id                 > t1.row_id
 WHERE NOT EXISTS (
SELECT 1
  FROM det                       d1
 WHERE d1.owner                  = t1.owner
   AND d1.table_name             = t1.table_name
   AND (
   NOT EXISTS (
SELECT 1
  FROM det                       d2
 WHERE d2.owner                  = t2.owner
   AND d2.table_name             = t2.table_name
   AND d2.r_owner                = d1.r_owner
   AND d2.r_constraint_name      = d1.r_constraint_name
   AND d2.n_dup                  = d1.n_dup
)))
 ORDER BY t1.owner,
       t1.table_name,
       t2.owner,
       t2.table_name

************
GTT_NE
************
  WITH tab AS (
SELECT t.owner,
       t.table_name,
       t.ROWID                   row_id,
       Sum (c.n_con)             n_det,
       Min (c.r_constraint_name) min_det,
       Max (c.r_constraint_name) max_det
  FROM tab_cp                    t
  JOIN grp_gtt                   c
    ON c.owner                   = t.owner
   AND c.table_name              = t.table_name
 GROUP BY
        t.owner,
        t.table_name,
        t.ROWID
)
SELECT
       t1.owner                  owner_1,
       t1.table_name             table_name_1,
       t2.owner                  owner_2,
       t2.table_name             table_name_2,
       t1.n_det                  n_det
  FROM tab                       t1
  JOIN tab                       t2
    ON t2.n_det                  = t1.n_det
   AND t2.min_det                = t1.min_det
   AND t2.max_det                = t1.max_det
   AND t2.row_id                 > t1.row_id
 WHERE NOT EXISTS (
SELECT 1
  FROM grp_gtt                   d1
 WHERE d1.owner                  = t1.owner
   AND d1.table_name             = t1.table_name
   AND (
   NOT EXISTS (
SELECT 1
  FROM grp_gtt                   d2
 WHERE d2.owner                  = t2.owner
   AND d2.table_name             = t2.table_name
   AND d2.r_owner                = d1.r_owner
   AND d2.r_constraint_name      = d1.r_constraint_name
   AND d2.n_con                  = d1.n_con
)))
 ORDER BY t1.owner,
       t1.table_name,
       t2.owner,
       t2.table_name

************
GTT_NE_X
************
  WITH tab AS (
SELECT t.owner,
       t.table_name,
       t.ROWID                   row_id,
       Sum (c.n_con)             n_det
  FROM tab_cp                    t
  JOIN grp_gtt                   c
    ON c.owner                   = t.owner
   AND c.table_name              = t.table_name
 GROUP BY
        t.owner,
        t.table_name,
        t.ROWID
)
SELECT
       t1.owner                  owner_1,
       t1.table_name             table_name_1,
       t2.owner                  owner_2,
       t2.table_name             table_name_2,
       t1.n_det                  n_det
  FROM tab                       t1
  JOIN tab                       t2
    ON t2.n_det                  = t1.n_det
   AND t2.row_id                 > t1.row_id
 WHERE NOT EXISTS (
SELECT 1
  FROM grp_gtt                   d1
 WHERE d1.owner                  = t1.owner
   AND d1.table_name             = t1.table_name
   AND (
   NOT EXISTS (
SELECT 1
  FROM grp_gtt                   d2
 WHERE d2.owner                  = t2.owner
   AND d2.table_name             = t2.table_name
   AND d2.r_owner                = d1.r_owner
   AND d2.r_constraint_name      = d1.r_constraint_name
   AND d2.n_con                  = d1.n_con
)))
 ORDER BY t1.owner,
       t1.table_name,
       t2.owner,
       t2.table_name

************
GTT_MI
************
  WITH tab AS (
SELECT t.owner,
       t.table_name,
       t.ROWID                   row_id,
       Sum (c.n_con) n_det,
       Min (c.r_constraint_name) min_det,
       Max (c.r_constraint_name) max_det
  FROM tab_cp                    t
  JOIN grp_gtt                   c
    ON c.owner                   = t.owner
   AND c.table_name              = t.table_name
 GROUP BY
        t.owner,
        t.table_name,
        t.ROWID
)
SELECT
       t1.owner                  owner_1,
       t1.table_name             table_name_1,
       t2.owner                  owner_2,
       t2.table_name             table_name_2,
       t1.n_det                  n_det
  FROM tab                       t1
  JOIN tab                       t2
    ON t2.n_det                  = t1.n_det
   AND t2.min_det                = t1.min_det
   AND t2.max_det                = t1.max_det
   AND t2.row_id                 > t1.row_id
 WHERE NOT EXISTS (
SELECT d2.r_owner,
       d2.r_constraint_name,
       d2.n_con
  FROM grp_gtt                   d2
 WHERE d2.owner                  = t2.owner
   AND d2.table_name             = t2.table_name
MINUS
SELECT d1.r_owner,
       d1.r_constraint_name,
       d1.n_con
  FROM grp_gtt                   d1
 WHERE d1.owner                  = t1.owner
   AND d1.table_name             = t1.table_name
)
 ORDER BY t1.owner,
       t1.table_name,
       t2.owner,
       t2.table_name

The query structure diagrams (QSDs) are in the embedded Excel file below:

Performance Analysis

We presented five query variations above, and in this section give the results of benchmarking these queries across a 1-dimensional data domain obtained by copying the system views 1, 2 and 4 times into my test tables described above. The problem sizes are as follows:
Record Counts

Timings and Statistics
Click on the query name in the file below to jump to the execution plan for the largest data point.

Comparison

The timings above are only for the main queries, so we need also to consider the time to populate and delete the temporary table, for the three GTT queries. This is performed as part of the data point setup, and the framework prints out timings for this part separately. For the last data point, the output was:

5144 records inserted in grp_gtt
8956 tables, 44780 constraints, 5 c/t

Timer Set: Setup, Constructed at 09 Oct 2012 22:28:49, written at 22:29:04
==========================================================================
[Timer timed: Elapsed (per call): 0.05 (0.000047), CPU (per call): 0.05 (0.000050), calls: 1000, '***' denotes corrected line below]

Timer                  Elapsed          CPU          Calls        Ela/Call        CPU/Call
-----------------   ----------   ----------   ------------   -------------   -------------
Delete test data          3.33         2.78              1         3.33200         2.78000
Delete GTT                0.18         0.17              1         0.18000         0.17000
Insert tab                0.55         0.44              1         0.54500         0.44000
Insert con                7.89         5.94              1         7.89000         5.94000
Insert grp_gtt            0.14         0.14              1         0.14000         0.14000
Count records             0.02         0.01              1         0.01600         0.01000
Gather statistics         2.59         2.06              1         2.58900         2.06000
(Other)                   0.00         0.00              1         0.00000         0.00000
-----------------   ----------   ----------   ------------   -------------   -------------
Total                    14.69        11.54              8         1.83650         1.44250
-----------------   ----------   ----------   ------------   -------------   -------------

The elapsed times for deleting from, then inserting into the temporary table are given by the 'Delete GTT' and 'Insert grp_gtt' timers and add up to 0.32s, so do not make much difference (about 5% on the best and less on the others). The following points can be made:

  • Doing the detail grouping and counting directly gives the worst performance
  • Moving the detail grouping and counting into a subquery factor improves performance by a factor of about 4
  • Moving the detail grouping into a temporary table improves performance the most
  • Using NOT EXISTS instead of MINUS for detail matching improves performance, as expected, by a factor of about 2
  • Using the minimum and maximum aggregates for pre-filtering, in addition to the counts, improves performance by a factor of about 20

Subquery Factors and Temporary Tables
If you look at the execution plan for INL_MI, most of the work is done in the HASH GROUP BY steps, 16 and 20, on totals of 127K records each. Moving this grouping into a subquery factor in SQF_NE means that the operation is done only once rather than many times (110K), and the execution plan for SUBQ_NE shows that it takes very little time (line 3).

However, the execution plan for SUBQ_NE shows (lines 14-22) that the factors are read using full scans, because indexes are not possible. This observation led to the improvement of moving the grouping out of the query altogether and into a separate stage that populates a temporary table, on which indexes can be defined. Lines 15-18 in the plan for GTT_NE show the access is now by index on the detail records.

Memory Usage in INL_MI Query with Grouping
Originally, I tried to have a larger range of data points, but doubling the size again always resulted in an Oracle error on INL_MI, ORA-04030: out of process memory when trying to allocate 123404 bytes (QERGH hash-agg,kllcqas:kllsltba). This is surprising because the execution plan statistics include memory statistics that appear to indicate that all queries use the same maximum amount of memory, which is just over 3MB, incurred in the SORT ORDER BY step (e.g. line 7 below).

My framework also collects statistics from the system view v$mystat, and prints out those showing large variations in 'after minus before' differences across the queries. The framework printed the statistic 'session pga memory' and this tells a different story (the values are in the embedded Excel files under Statistics above). The INL_MI query shows increases of 14MB, then 170MB, then 768MB approx. while the other queries all show no increases. It's hard to understand what's going on here, but one guess is that the query is revealing an Oracle bug that causes memory not to be released after use and then re-used, but for new executions of the relevant operation to request new memory, and that the execution plans are not reporting this. However, as discussed later, the variation in execution time with problem size is also difficult to understand and suggests that the HASH GROUP BY operations are really being performed on the entire record sets, which would also greatly increase the memory usage. The version, running under Windows 7 is: Oracle Database 11g Express Edition Release 11.2.0.2.0 - Beta

Execution Plan Hash Values
In my last article, I was able to easily identify the distinct plans by looking at the matrix of plan hash values, with values the same indicating the same plan. This time though, that doesn't work: all hash values are different, but in fact, inspection of the plans shows that for each query there was essentially only one plan. I believe this may be due to the subquery factors, which result in a system-generated view name, which differs each time. For example, here are the last two plans for the INL_MI, where the only difference appears to be in the view names (SYS_TEMP_0FD9D6681_1B0CFB4 for W2 and SYS_TEMP_0FD9D6687_1B0CFB4 for W4) (note that different statistics don't make the plans different):

Point W2:

Plan hash value: 89269728

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------
| Id  | Operation                         | Name                       | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  | Writes |  OMem |  1Mem | Used-Mem |
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT                  |                            |      1 |        |   8054 |00:01:40.96 |     191K|     21 |     21 |       |       |          |
|   1 |  TEMP TABLE TRANSFORMATION        |                            |      1 |        |   8054 |00:01:40.96 |     191K|     21 |     21 |       |       |          |
|   2 |   LOAD AS SELECT                  |                            |      1 |        |      0 |00:00:00.05 |    3182 |      0 |     21 |   264K|   264K|  264K (0)|
|   3 |    HASH GROUP BY                  |                            |      1 |   2606 |   1628 |00:00:00.05 |    3158 |      0 |      0 |   766K|   766K| 1273K (0)|
|   4 |     NESTED LOOPS                  |                            |      1 |   2606 |   2606 |00:00:00.05 |    3158 |      0 |      0 |       |       |          |
|*  5 |      TABLE ACCESS FULL            | CON_CP                     |      1 |   2606 |   2606 |00:00:00.01 |     625 |      0 |      0 |       |       |          |
|*  6 |      INDEX UNIQUE SCAN            | TCP_PK                     |   2606 |      1 |   2606 |00:00:00.02 |    2533 |      0 |      0 |       |       |          |
|   7 |   SORT ORDER BY                   |                            |      1 |      1 |   8054 |00:01:40.89 |     188K|     21 |      0 |  1824K|   650K| 1621K (0)|
|*  8 |    FILTER                         |                            |      1 |        |   8054 |00:01:24.17 |     188K|     21 |      0 |       |       |          |
|*  9 |     HASH JOIN                     |                            |      1 |      1 |  27242 |00:00:00.15 |      47 |     21 |      0 |   720K|   720K| 1282K (0)|
|  10 |      VIEW                         |                            |      1 |   2606 |   1628 |00:00:00.01 |      25 |     21 |      0 |       |       |          |
|  11 |       TABLE ACCESS FULL           | SYS_TEMP_0FD9D6681_1B0CFB4 |      1 |   2606 |   1628 |00:00:00.01 |      25 |     21 |      0 |       |       |          |
|  12 |      VIEW                         |                            |      1 |   2606 |   1628 |00:00:00.01 |      22 |      0 |      0 |       |       |          |
|  13 |       TABLE ACCESS FULL           | SYS_TEMP_0FD9D6681_1B0CFB4 |      1 |   2606 |   1628 |00:00:00.01 |      22 |      0 |      0 |       |       |          |
|  14 |     MINUS                         |                            |  27242 |        |  19188 |00:01:40.05 |     188K|      0 |      0 |       |       |          |
|  15 |      SORT UNIQUE                  |                            |  27242 |      1 |  28722 |00:00:53.68 |   73872 |      0 |      0 |  2048 |  2048 | 2048  (0)|
|  16 |       HASH GROUP BY               |                            |  27242 |      1 |  31314 |00:00:45.36 |   73872 |      0 |      0 |   750K|   750K|  610K (0)|
|* 17 |        TABLE ACCESS BY INDEX ROWID| CON_CP                     |  27242 |      1 |  31335 |00:00:01.57 |   73872 |      0 |      0 |       |       |          |
|* 18 |         INDEX RANGE SCAN          | CON_TAB_FK_N1              |  27242 |      1 |    175K|00:00:01.26 |   42504 |      0 |      0 |       |       |          |
|  19 |      SORT UNIQUE                  |                            |  27242 |      1 |  30502 |00:00:45.94 |     114K|      0 |      0 |  2048 |  2048 | 2048  (0)|
|  20 |       HASH GROUP BY               |                            |  27242 |      1 |  31314 |00:00:37.86 |     114K|      0 |      0 |   750K|   750K|  910K (0)|
|* 21 |        TABLE ACCESS BY INDEX ROWID| CON_CP                     |  27242 |      1 |  31335 |00:00:01.64 |     114K|      0 |      0 |       |       |          |
|* 22 |         INDEX RANGE SCAN          | CON_TAB_FK_N1              |  27242 |      1 |    183K|00:00:01.29 |   83068 |      0 |      0 |       |       |          |
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   5 - filter("C"."CONSTRAINT_TYPE"='R')
   6 - access("C"."OWNER"="T"."OWNER" AND "C"."TABLE_NAME"="T"."TABLE_NAME")
   8 - filter( IS NULL)
   9 - access("T2"."N_DET"="T1"."N_DET" AND "T2"."MIN_DET"="T1"."MIN_DET" AND "T2"."MAX_DET"="T1"."MAX_DET")
       filter("T2"."ROW_ID">"T1"."ROW_ID")
  17 - filter("C2"."CONSTRAINT_TYPE"='R')
  18 - access("C2"."OWNER"=:B1 AND "C2"."TABLE_NAME"=:B2)
  21 - filter("C1"."CONSTRAINT_TYPE"='R')
  22 - access("C1"."OWNER"=:B1 AND "C1"."TABLE_NAME"=:B2)

Point W4:

Plan hash value: 892071883

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------
| Id  | Operation                         | Name                       | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  | Writes |  OMem |  1Mem | Used-Mem |
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT                  |                            |      1 |        |  16108 |00:25:33.98 |     788K|     42 |     42 |       |       |          |
|   1 |  TEMP TABLE TRANSFORMATION        |                            |      1 |        |  16108 |00:25:33.98 |     788K|     42 |     42 |       |       |          |
|   2 |   LOAD AS SELECT                  |                            |      1 |        |      0 |00:00:00.10 |    5802 |      0 |     42 |   521K|   521K|  521K (0)|
|   3 |    HASH GROUP BY                  |                            |      1 |   5007 |   3256 |00:00:00.09 |    5757 |      0 |      0 |  1001K|   943K| 1273K (0)|
|   4 |     NESTED LOOPS                  |                            |      1 |   5007 |   5212 |00:00:00.09 |    5757 |      0 |      0 |       |       |          |
|*  5 |      TABLE ACCESS FULL            | CON_CP                     |      1 |   4980 |   5212 |00:00:00.01 |     625 |      0 |      0 |       |       |          |
|*  6 |      INDEX UNIQUE SCAN            | TCP_PK                     |   5212 |      1 |   5212 |00:00:00.04 |    5132 |      0 |      0 |       |       |          |
|   7 |   SORT ORDER BY                   |                            |      1 |      1 |  16108 |00:25:33.84 |     782K|     42 |      0 |  3596K|   822K| 3196K (0)|
|*  8 |    FILTER                         |                            |      1 |        |  16108 |00:22:30.61 |     782K|     42 |      0 |       |       |          |
|*  9 |     HASH JOIN                     |                            |      1 |      1 |    110K|00:00:00.62 |      89 |     42 |      0 |   900K|   900K| 1328K (0)|
|  10 |      VIEW                         |                            |      1 |   5007 |   3256 |00:00:00.02 |      46 |     42 |      0 |       |       |          |
|  11 |       TABLE ACCESS FULL           | SYS_TEMP_0FD9D6687_1B0CFB4 |      1 |   5007 |   3256 |00:00:00.01 |      46 |     42 |      0 |       |       |          |
|  12 |      VIEW                         |                            |      1 |   5007 |   3256 |00:00:00.03 |      43 |      0 |      0 |       |       |          |
|  13 |       TABLE ACCESS FULL           | SYS_TEMP_0FD9D6687_1B0CFB4 |      1 |   5007 |   3256 |00:00:00.02 |      43 |      0 |      0 |       |       |          |
|  14 |     MINUS                         |                            |    110K|        |  94488 |00:25:29.91 |     782K|      0 |      0 |       |       |          |
|  15 |      SORT UNIQUE                  |                            |    110K|      1 |    113K|00:14:20.41 |     300K|      0 |      0 |  2048 |  2048 | 2048  (0)|
|  16 |       HASH GROUP BY               |                            |    110K|      1 |    127K|00:11:47.70 |     300K|      0 |      0 |   789K|   789K|  527K (0)|
|* 17 |        TABLE ACCESS BY INDEX ROWID| CON_CP                     |    110K|      1 |    127K|00:00:08.15 |     300K|      0 |      0 |       |       |          |
|* 18 |         INDEX RANGE SCAN          | CON_TAB_FK_N1              |    110K|      1 |    722K|00:00:06.55 |     156K|      0 |      0 |       |       |          |
|  19 |      SORT UNIQUE                  |                            |    110K|      1 |    123K|00:11:07.57 |     481K|      0 |      0 |  2048 |  2048 | 2048  (0)|
|  20 |       HASH GROUP BY               |                            |    110K|      1 |    127K|00:09:52.37 |     481K|      0 |      0 |   789K|   789K|  907K (0)|
|* 21 |        TABLE ACCESS BY INDEX ROWID| CON_CP                     |    110K|      1 |    127K|00:00:08.37 |     481K|      0 |      0 |       |       |          |
|* 22 |         INDEX RANGE SCAN          | CON_TAB_FK_N1              |    110K|      1 |    735K|00:00:06.31 |     337K|      0 |      0 |       |       |          |
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   5 - filter("C"."CONSTRAINT_TYPE"='R')
   6 - access("C"."OWNER"="T"."OWNER" AND "C"."TABLE_NAME"="T"."TABLE_NAME")
   8 - filter( IS NULL)
   9 - access("T2"."N_DET"="T1"."N_DET" AND "T2"."MIN_DET"="T1"."MIN_DET" AND "T2"."MAX_DET"="T1"."MAX_DET")
       filter("T2"."ROW_ID">"T1"."ROW_ID")
  17 - filter("C2"."CONSTRAINT_TYPE"='R')
  18 - access("C2"."OWNER"=:B1 AND "C2"."TABLE_NAME"=:B2)
  21 - filter("C1"."CONSTRAINT_TYPE"='R')
  22 - access("C1"."OWNER"=:B1 AND "C1"."TABLE_NAME"=:B2)

Performance Variation Polynomials
The timings above show that CPU and elapsed times increased by different powers of the problem size increases, according to query.

The inline grouping query INL_MI shows a variation close to the fourth power, which like its memory usage, is very hard to understand. Most of the time is used in the HASH GROUP BY operations at lines 16 and 20, and it rises about 16 times betwen W2 and W4. The numbers of starts rise by 4 times, as expected, but the number of rows per start remains constant at about 1.15, so the work done should rise by about 4 times. It's almost as though the SQL engine is really processing the entire record set in the HASH GROUP BY, rather than just the subset for the correlated tables, contrary to what the plan says. Again, this looks buggy.

The double subquery factor query SUBQ_NE has about a cubic variation, which is plausible because the table pairing introduces a quadratic term, with the third power coming from the full scans of the detail subquery factor.

All three of the temporary table queries show quadratic variation, which is likely the best obtainable while matching sets directly (but see my next article Holographic Set Matching in SQL for linear solutions bypassing set matching), and arises from the table pairing, but with the details for each pair being constant in size and accessed via indexes. It's worth noting that the query GTT_NE_X is actually slower than INL_MI and SUB_NE on the smallest data point, but much quicker on the largest, showing the importance of polynomial order for scalability.

Conclusions

  • We have shown how to solve master-detail transaction matching problems efficiently, using an example problem, but emphasising the generality of the techniques
  • Appropriate use of subquery factors and temporary tables have been demonstrated, with performance analysis
  • It's worth highlighting the technique of pre-filtering on aggregates before comparing sets in detail
  • The importance for scalability of performance variation powers has been illustrated, being revealed by dimensional benchmarking
  • On finishing this article it occurred to me to wonder whether it might not be possible to use aggregate matching to replace detail set matching altogether, and at least in some cases it is, with linear performance resulting, described in my next article, Holographic Set Matching in SQL (MDTM2)