DanHeidel DanHeidel - 2 years ago 69
SQL Question

SQL, querying by date intervals

I've got a dataset that I want to be able to slice up by date interval. It's a bunch of scraped web data and each item has a unix-style milisecond timestamp as well as a standard UTC datetime.

I'd like to be able to query the dataset, picking out the rows that are closest to various time intervals:
e.g.: Every hour, once a day, once a week, etc.

There is no guarantee that the timestamps are going to fall evenly on the interval times, otherwise I'd just do a mod query on the timestamp.

Is there a way to do this with SQL commands that doesn't involve stored procs or some sort of pre-computed support tables?

I use the latest MariaDB.

The marked answer doesn't quite answer my specific question but it is a decent answer to the more generalized problem so I went ahead and marked it.

I was specifically looking for a way to query a set of data where the timestamp is highly variable and to grab out rows that are reasonably close to periodic time intervals. E.g.: get all the rows that are the closest to being on 24 hour intervals from right now.

I ended up using a modulus query to solve the problem: timestamp % interval < average spacing between data points. This occasionally grabs extra points and misses a few but was good enough for my graphing application.

And them I got sick of the node-mysql library crashing all the time so I moved to MongoDB.

Answer Source

You say you want 'closest to various time intervals' but then say 'every hour/day/week', so the actual implementation will depend on what you really want, but you can use a host of standard date/time functions to group records, for example count by day:

SELECT DATE(your_DateTime) AS Dt, COUNT(something) AS CT
FROM yourTable
GROUP BY DATE(your_DateTime)

Count by Hour:

SELECT DATE(your_DateTime) AS Dt,HOUR(your_DateTime) AS Hr, COUNT(something) AS CT
FROM yourTable
GROUP BY DATE(your_DateTime), HOUR(your_DateTime)

See the full list of supported date and time functions here: https://mariadb.com/kb/en/date-and-time-functions/

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download