hax0r_n_code hax0r_n_code - 1 month ago 13
Python Question

with open(file, 'a+') as f is not appending

I have a function that gets called repetitively and writes error data to a log file via

subprocess
, however, whenever new data is written to the file, the old data is cleared. So it is not appending any new data with the old data.

I'm doing following:

error_file = '\\\\some\\unc\\path\\error.log'

class Node:
def __init__(self, path, rev):
self.path = path
self.rev = rev

def __hash__(self):
return hash((self.path, self.rev))

def __eq__(self, node):
return (self.path, self.rev) == (node.path, node.rev)

def __ne__(self, node):
return not(self == node)

def get_excluded_nodes(excludes_dir):
nodes = list()
for root, subdirs, files in os.walk(os.path.dirname(excludes_dir)):
if 'flagged' in files:
with open(os.path.join(root, 'flagged')) as f:
for line in f.readlines():
try:
comps = line.split(' -a')
path = comps[0].strip()
rev = comps[1].split(':')[0].strip()
Nodes.append(Node(path,rev))
except:
pass
return nodes

def export_node(node, path=archive_dir):
with open(error_file, 'a') as f:
try:
comps = node.path.split('/')
if '.' in comps[len(comps)-1]:
os.makedirs(os.path.join(archive_dir, '/'.join(comps[:-1])))
else:
os.makedirs(os.path.join(archive_dir, node.path))

subprocess.call(['svn', 'export', os.path.join(some_path, node.path), another_path)], stderr=f)
except:
pass


def remove_duplicate_nodes(nodes):
return set(nodes)

if __name__ == '__main__':
all_nodes = get_excluded_nodes(os.path.realpath(__file__))
nodes = remove_duplicate_nodes(all_nodes)
for node in nodes:
export_node(node)


Why doesn't this work?

Answer

That's still not an MVCE: it isn't minimal (what's Node for, how does it affect appending to a file?) and it isn't complete (where is get_excluded_nodes?). Since it isn't complete, it can't be verifiable either.

This is an MVCE with the minimal code that ought to do the same thing as your example, and it works fine.

import subprocess

def test(filename):
    with open(filename, 'a') as f:
        subprocess.call(['bash', '-c', 'echo $$ >&2'], stderr=f)

if __name__=='__main__':
    for _ in range(2):
        test('stderr.log')

which does exactly what you wanted:

$ python stderr.py 
$ cat stderr.log 
344
345

$ python stderr.py 
$ cat stderr.log 
344
345
366
367

edit I see you're on Windows, so strace is probably out. Bad luck, you'll just have to write an actual MVCE instead.


Maybe run your original (real, sort-of-working) script under strace, and see what's different. For reference, this script shows:

$ strace -f -e trace=open,dup2,lseek,write python stderr.py

open("stderr.log", O_WRONLY|O_CREAT|O_APPEND, 0666) = 3
lseek(3, 0, SEEK_END)                   = 28
strace: Process 567 attached
...
[pid   567] dup2(3, 2)                  = 2 # stderr=f
... libc, locale stuff ...
[pid   567] dup2(2, 1)                  = 1 # >&2
[pid   567] write(1, "567\n", 4)        = 4
Comments