hgus1294 hgus1294 - 6 months ago 41
SQL Question

Java MySQL Hibernate slow select queries

I have a MySQL table for persistence of some Java objects and use Hibernate for mapping of data/columns. The objects/table contain around 50 columns/fields. On the surface and under minimal load this works fine but I have run into problems on startup of the program where I load all available objects from the table. There is currently around 8500 rows in the table. The loading of these rows takes painfully long, in excess of 5 minutes!. Obviously this isn't right and I must have done something horribly wrong in my Hibernate configuration (or elsewhere). A normal query for all rows using the java.sql.* package takes around 300 ms to retrieve the 8000+ rows which is acceptable, but ideally I would like to keep Hibernate for its mapping functionality.

I have attached the main configuration for Hibernate below. I am happy to provide further configuration/implementation details but I do not want to bloat this question with what may be unnecessary information at this stage.

<hibernate-configuration>
<session-factory>
<property name="hibernate.connection.driver_class">
com.mysql.jdbc.Driver</property>
<property name="hibernate.connection.url">

jdbc:mysql://127.0.0.1:3306/abc</property>
<property name="hibernate.connection.username">user</property>
<property name="hibernate.connection.password">password</property>

<property name="org.hibernate.SQL">true</property>
<property name="dialect">org.hibernate.dialect.MySQLDialect</property>
<property name="hibernate.hbm2ddl.auto">update</property>

<property name="hibernate.c3p0.idle_test_period">60</property>
<property name="hibernate.c3p0.min_size">5</property>
<property name="hibernate.c3p0.max_size">20</property>
<property name="hibernate.c3p0.max_statements">250</property>
<property name="hibernate.c3p0.timeout">1800</property>
<property name="hibernate.c3p0.acquireRetryAttempts">1</property>
<property name="hibernate.c3p0.acquireRetryDelay">250</property>

<property name= "hibernate.transaction.factory_class">
org.hibernate.transaction.JDBCTransactionFactory
</property>
<property name="hibernate.current_session_context_class">thread</property>


I was originally using the default connection pool but tried switching to C3P0 in hope that it would help but the loading time of the rows is basically the same regardless of pool.

Obviously I am anything but a Hibernate expert so I would really appreciate some insight to what I am doing wrong. Thanks.

UPDATE1
Adding log output from the queries.

20:16:40,633 DEBUG QueryTranslatorImpl:283 - --- HQL AST ---
\-[QUERY] Node: 'query'
+-[SELECT_FROM] Node: 'SELECT_FROM'
| \-[FROM] Node: 'from'
| \-[RANGE] Node: 'RANGE'
| \-[DOT] Node: '.'
| +-[DOT] Node: '.'
| | +-[DOT] Node: '.'
| | | +-[DOT] Node: '.'
| | | | +-[IDENT] Node: 'com'
| | | | \-[IDENT] Node: 'xyz'
| | | \-[IDENT] Node: 'objects'
| | \-[IDENT] Node: 'abc'
| \-[IDENT] Node: 'ObjectID'
\-[WHERE] Node: 'where'
\-[EQ] Node: '='
+-[IDENT] Node: 'event_id'
\-[NUM_INT] Node: '1331570489282'

20:16:40,633 DEBUG QueryTranslatorImpl:252 - --- SQL AST ---
\-[SELECT] QueryNode: 'SELECT' querySpaces (order_log)
+-[SELECT_CLAUSE] SelectClause: '{derived select clause}'
| +-[SELECT_EXPR] SelectExpressionImpl: 'objectid0_.event_id as abc1_0_' {FromElement{explicit,not a collection join,not a fetch join,fetch non-lazy properties,classAlias=null,role=null,tableName=order_log,tableAlias=objectid0_,origin=null,columns={,className=com.xyz.objects.abc.objectid}}}
| \-[SQL_TOKEN] SqlFragment: 'objectid0_.order_id as abc2_0_, objectid0_.order_id as order3_0_, objectid0_.px_cond as px4_0_, objectid0_.px_curr as px5_0_, objectid0_.px_value as px6_0_, objectid0_.vol_cond as vol7_0_, objectid0_.vol as vol0_, objectid0_.side as side0_, objectid0_.trd_vol as trd10_0_, objectid0_.open_vol as open11_0_, objectid0_.accno as accno0_, objectid0_.symbol as symbol0_, objectid0_.market as market0_, objectid0_.validity_type as validity15_0_, objectid0_.validity_date as validity16_0_, objectid0_.mod_date as mod17_0_, objectid0_.result_code as result18_0_, objectid0_.order_state as order19_0_, objectid0_.action_state as action20_0_, objectid0_.action_type as action21_0_, objectid0_.action_status as action22_0_, objectid0_.status_text as status23_0_, objectid0_.strategy_id as strategy24_0_, objectid0_.passive as passive0_, objectid0_.timestamp as timestamp0_, objectid0_.confirmedorder as confirm27_0_, objectid0_.updatesource as updates28_0_'
+-[FROM] FromClause: 'from' FromClause{level=1, fromElementCounter=1, fromElements=1, fromElementByClassAlias=[], fromElementByTableAlias=[objectid0_], fromElementsByPath=[], collectionJoinFromElementsByPath=[], impliedElements=[]}
| \-[FROM_FRAGMENT] FromElement: 'order_log objectid0_' FromElement{explicit,not a collection join,not a fetch join,fetch non-lazy properties,classAlias=null,role=null,tableName=order_log,tableAlias=objectid0_,origin=null,columns={,className=com.abc.objects.xyz.objectid}}
\-[WHERE] SqlNode: 'where'
\-[EQ] BinaryLogicOperatorNode: '='
+-[IDENT] IdentNode: 'event_id' {originalText=event_id}
\-[NUM_INT] LiteralNode: '1331570489282'


There is a lot more output in the logs but I figured that the 2 items above were the most relevant. If I have missed anything important, please let me know.

UPDATE2 Adding the actual sql-output.

Hibernate: /* from objectid where abc_event_id=1331570505614 */ select objectid0
_.abc_event_id as abc1_0_, objectid0_.abc_order_id as abc2_0_, objectid0_.
order_id as order3_0_, objectid0_.px_cond as px4_0_, objectid0_.px_curr as px5_0_,
objectid0_.px_value as px6_0_, objectid0_.vol_cond as vol7_0_, objectid0_.vol as v
ol0_, objectid0_.side as side0_, objectid0_.trd_vol as trd10_0_, objectid0_.open_vo
l as open11_0_, objectid0_.accno as accno0_, objectid0_.symbol as symbol0_, ucorde
r0_.market as market0_, objectid0_.validity_type as validity15_0_, objectid0_.vali
dity_date as validity16_0_, objectid0_.mod_date as mod17_0_, objectid0_.result_cod
e as result18_0_, objectid0_.order_state as order19_0_, objectid0_.action_state as
action20_0_, objectid0_.action_type as action21_0_, objectid0_.action_status as a
ction22_0_, objectid0_.status_text as status23_0_, objectid0_.strategy_id as strat
egy24_0_, objectid0_.passive as passive0_, objectid0_.timestamp as timestamp0_, uc
order0_.confirmedorder as confirm27_0_, objectid0_.updatesource as updates28_0_ f
rom order_log objectid0_ where abc_event_id=1331570505614


UPDATE 3 Object Mapping + Transaction code

<hibernate-mapping>
<class name="com.abc.objects.xyz.objectid" table="order_log">
<id name="EventId" type="long" column="event_id" >
<generator class="assigned"/>
</id>

<property name="ordId">
<column name="order_id"/>
</property>
<property name="pxCond">
<column name="px_cond"/>
</property>
...
</hibernate-mapping>


The mapping is an excerpt but it is straight forward along the same lines as above. The Transaction that queries the table looks like below:

Session session = HibernateUtil.getSessionFactory().openSession();
org.hibernate.Transaction transaction = null;
try {
long t0 = System.currentTimeMillis();
transaction = session.beginTransaction();
List<UCOrder> result = session.createQuery("from UCOrder").list();
transaction.commit();
System.out.println (result.size() + " rows were retrieved");
System.out.println("Time elapsed: " + (System.currentTimeMillis()-t0));
} catch (Exception e) {
transaction.rollback();
e.printStackTrace();
} finally {
session.close();
}


Also rather straight forward I would like to think?!?. However, the time it takes to retrieve the data is thousandfold compared to the vanilla
java.sql.*
method illustrated below:

Connection conn = DriverManager.getConnection (url, userName, password);
Statement s = conn.createStatement ();
s.executeQuery ("SELECT * FROM order_log");
ResultSet rs = s.getResultSet ();

Answer

I suggest you switching on a verbose logging in Hibernate. You can get it to log all SQL queries and see if there is something unusual in them.

One way to do this is setting hibernate.show_sql to true and/or (?) configure commons-logging used by Hibernate to output all org.hibernate category to your favourite output file.

Comments