lonerangerdeveloper lonerangerdeveloper - 1 year ago 69
Linux Question

Using a command's output as `paste` input without temporary files

I am new to bash scripting and I have what I assume a very novice question.

How would I go about concatenating the output of the following 3 commands into a single line of code?

The line below gets the current system date minus 1 day (24hrs) and any files greater than current date -1 day get written to redir_bbtrm_files.txt

awk -v date="$(date -d "-1 day" +"%Y-%m-%d %H:%M")" '$3$4<date' sftp_bbtrm_files.txt \
> redir_bbtrm_files.txt

The line below prints date, time and file name from redir_bbtrm_files.txt and redirects output to bbtrm_filenames_n_time.txt

awk '{print $3,$4,$5}' redir_bbtrm_files.txt > bbtrm_filenames_n_time.txt

The line below appends file names, time, and reference numbers. email_final_bbtrm_output.txt is the output that will be in the notification

paste -d ' ' bbtrm_filenames_n_time.txt bbtrm_ref.txt > email_final_bbtrm_output.txt

Basically, I am using the commands above to save off the output of each command in a file and then appending the results at the end. But I would imagine that this is not a good way to do this and there must be some other way to handle this scenario in a more efficient way.

Can the above be done in 1 line of code and maybe save off the output into a single file?

Answer Source

Easily done as a single command, assuming your shell is bash. Assuming that I'm reading your question correctly:

#      ^^^^- NOT /bin/sh; also must not be run with "sh yourscript" (use "bash yourscript")

paste -d ' ' \
  <(awk -v date="$(date -d "-1 day" +"%Y-%m-%d %H:%M")" '$3$4<date' sftp_bbtrm_files.txt \
    | awk '{print $3,$4,$5}') \
  bbtrm_ref.txt \

To explain:

  • foo | bar connects the stdout of foo to the stdin of bar; thus, it can be used to send the output of your first awk to your second one (though why you need two separate awk commands there rather than only one is a bit unclear).
  • <(...) is an expansion which is replaced with a filename (either a named pipe or a /dev/fd reference) which, when read, returns the output of the command ...

Thus, we're pasteing together both bbtrm_ref.txt and the result of sending sftp_bbtrm_files.txt through your two awk commands to get your single output file.