Pynoob Pynoob - 2 months ago 7
Linux Question

python + linux + not killing subprocess issues

I am trying to find out a list of files within a directory path, which were created within the past 'n' minutes, using subprocess.popen with 'find' linux command.

def Subprocessroutine(n,path):
argf='find '+str(path)+' -maxdepth 1 -mmin -'+str(n)+' -type f -printf "\n%AD %AT %p" | sort'
p=subprocess.Popen(argf,stdout=subprocess.PIPE,shell=True)
soutput,sinput=p.communicate()
return soutput


Since i am using a simple find command (with formatting and sort), do i need to make sure i kill the subprocess explicitly?.

The reason i am using the 'find' command is :
I need a list of files created in the past 'n' minutes and the result should in the file creation sorted order (I am unable to get this using the following code):

for p, ds, fs in os.walk(dirpath):
for fn in fs:
filepath = os.path.join(p, fn)
if os.path.getmtime(filepath) >= n:
print(datetime.datetime.fromtimestamp(os.path.getmtime(filepath)))


Is there a better way to get the result, without using the subprocess?.

My final output should look like this:
09/26/1619:41:04.4865673390 /home/Testing/Input/Sep26/HST39_2016-09-26 19:40:03.283121_2016-09-26 19:41:03.283109.csv
09/26/1619:41:04.4875673570 /home/Testing/Input/Sep26/HST40_2016-09-26 19:40:03.283561_2016-09-26 19:41:03.283552.csv
09/26/1619:41:04.4885673750 /home/Testing/Input/Sep26/HST41_2016-09-26 19:40:03.283988_2016-09-26 19:41:03.283980.csv
09/26/1619:41:04.4895673930 /home/Testing/Input/Sep26/HST42_2016-09-26 19:40:03.284408_2016-09-26 19:41:03.284399.csv
09/26/1619:41:04.4905674110 /home/Testing/Input/Sep26/HST43_2016-09-26 19:40:03.284852_2016-09-26 19:41:03.284843.csv
09/26/1619:41:04.4915674290 /home/Testing/Input/Sep26/HST44_2016-09-26 19:40:03.285295_2016-09-26 19:41:03.285288.csv

Answer

Based on your code, the only thing that "did not work" was the fact that dead links cause it to throw an OSError. I have modified it to the following:

#!/usr/bin/env python

import os
import time
import datetime

dirpath = "./"

for p, ds, fs in os.walk(dirpath):
    for fn in fs:
        filepath = os.path.join(p, fn)
        # This catches bad links!
        try:
            if os.path.getmtime(filepath) >= time.time() - 30*60:
                dt_mod = datetime.datetime.fromtimestamp(os.path.getmtime(filepath))
                print("Adding   %s - %s" % (str(dt_mod), filepath))
            else:
                print("Skipping %s - %s" % (str(dt_mod), filepath))
        except OSError as e:
            print("Arrrgggg: %s" % str(e))

Note that time.time() - 30*60 is current time minus 30 minutes, where time.time() gives you the current time as float. I am running in /tmp folder... the result is:

$ python ./test.py 
Adding   2016-09-26 19:48:24.913770 - ./test.py
Skipping 2016-09-26 19:48:24.913770 - ./qtsingleapp-homeur-6219-3e8-lockfile
Skipping 2016-09-26 19:48:24.913770 - ./.X0-lock
Skipping 2016-09-26 19:48:24.913770 - ./urban.socket
Skipping 2016-09-26 19:48:24.913770 - ./config-err-yVuRIy
Skipping 2016-09-26 19:48:24.913770 - ./qtsingleapp-homeur-6219-3e8
Adding   2016-09-26 19:19:53.685688 - ./.com.vivaldi.Vivaldi.82Dchr/SingletonSocket
Arrrgggg: [Errno 2] No such file or directory: './.com.vivaldi.Vivaldi.82Dchr/SingletonCookie'
Skipping 2016-09-26 19:19:53.685688 - ./ksocket-urban/klauncherhX3020.slave-socket
Skipping 2016-09-26 19:19:53.685688 - ./ksocket-urban/KSMserver__0
Skipping 2016-09-26 19:19:53.685688 - ./ksocket-urban/kdeinit4__0
Skipping 2016-09-26 19:19:53.685688 - ./skype-3294/DbTemp./temp-vL1IkSGDS19DRPp35idqPr9i
Skipping 2016-09-26 19:19:53.685688 - ./skype-3294/DbTemp./temp-oBEOkqraeZoXoyOrtvgzyxgE
Skipping 2016-09-26 19:19:53.685688 - ./skype-3294/DbTemp./temp-urolswo9ukl5kCzrRAD3epUw
Skipping 2016-09-26 19:19:53.685688 - ./skype-3294/DbTemp./temp-u5xJMdmkGEwV7K1nhqzx9Jg0
Adding   2016-09-26 19:44:48.309759 - ./kde-urban/konsole-Jw4443.history
Adding   2016-09-26 19:48:25.661770 - ./kde-urban/konsole-tX4539.history
Adding   2016-09-26 19:48:25.661770 - ./kde-urban/konsole-SK4539.history
Adding   2016-09-26 19:48:25.661770 - ./kde-urban/konsole-Jt4539.history
Skipping 2016-09-26 19:48:25.661770 - ./kde-urban/xauth-1000-_0
Adding   2016-09-26 19:44:48.309759 - ./kde-urban/konsole-cq4443.history
Adding   2016-09-26 19:44:48.309759 - ./kde-urban/konsole-WK4443.history
Skipping 2016-09-26 19:44:48.309759 - ./gpg-t0jSN1/S.gpg-agent
Skipping 2016-09-26 19:44:48.309759 - ./.X11-unix/X0
Skipping 2016-09-26 19:44:48.309759 - ./.ICE-unix/3044

I am not sorting or storing the results but this is easily doable...

EDIT

The sorting part involves storing the results that we like and printing them after sorting:

#!/usr/bin/env python

import os
import time
import datetime

dirpath = "./"

results = []

for p, ds, fs in os.walk(dirpath):
    for fn in fs:
        filepath = os.path.join(p, fn)
        # This catches bad links!
        try:
            if os.path.getmtime(filepath) >= time.time() - 30*60:
                dt_mod = os.path.getmtime(filepath)
                print("Adding   %s - %s" % (str(dt_mod), filepath))
                results.append((filepath, os.path.getmtime(filepath)))
            else:
                print("Skipping %s - %s" % (str(dt_mod), filepath))
        except OSError as e:
            print("Arrrgggg: %s" % str(e))


results.sort(key=lambda x: x[1])
for s in results:
    print(s)

I have removed datetime conversions since the result can be produced using only epoch timestamps

Comments