EcoWarrior EcoWarrior - 29 days ago 12
Python Question

Passing a date string into a function using pandas read_gbq is not recognized

Hello and thanks for taking the time to read....
The date string(i.e., date_start) appears to have been passed successfully into the function based on the two print functions' output(follows the code below) but it is not recognized as a date in the where clause since the result set is null (result set is not null outside of the function when using the hard-coded date string literals.

date_start = '2016-10-14 09:00:00'
def get_recs_w_date_range(date_l):
"Iterate through various date ranges to create the a timeframe sample for later aggregation"
global df1
global date_g
date_g = date_l
df1 = pd.read_gbq("select Timestamp, bytes, dst_addr, cast(dst_port as integer) as dst_port, \
cast(duration_ms as integer) as duration_ms, protocol, flow_direction \
FROM ipfix.ipfix \
where Timestamp between timestamp('date_l') and timestamp('2016-10-14 09:00:30') limit 50",
project_id="network-sec-analytics")
print('The value of local var date_l is: {}'.format(date_l))
return

get_recs_w_date_range(date_start)

print('The value of global var date_g is: {}'.format(date_g))

df1

The value of local var date_l is: 2016-10-14 09:00:00
The value of global var date_g is: 2016-10-14 09:00:00

Answer

Ok, I think it is an easy enough fix. You need to impute the value of date_l to your query, which you are not doing. Try this:

def get_recs_w_date_range(date_l):
    """Iterate through various date ranges to create the a timeframe sample for later aggregation"""
    query = """SELECT Timestamp, bytes, dst_addr,
                 cast(dst_port as integer) as dst_port,
                 cast(duration_ms as integer) as duration_ms,
                 protocol, flow_direction 
               FROM ipfix.ipfix
               WHERE Timestamp BETWEEN timestamp('{}') AND
                 timestamp('2016-10-14 09:00:30')
               LIMIT 50""".format(date_l)
    return pd.read_gbq(query, project_id="network-sec-analytics")

The format method will replace the {} with the value of date_l. Your query should work now.

Comments