If you cannot install non-standard software (example: customer owner, company operated RHEL boxes) you can get an idea of the current progress of cp/tar and similar programs by inspecting file-descriptor files in /proc.

Assuming that you have a process (like cp) with pid 1234, then in /proc/1234/fd you will find files with a number (corresponding to the file descriptor number in the context of the process) that are either symbolic links to the actual file or to socket connection (or other data structures). Let's assume that source file has fd 3 (you get this information via ls -l /proc/1234/fd )

Then, after you can cat /proc/1234/fdinfo/3 and look at the pos value: that's the position of the cursor in the file.

It's ugly and unpractical, but it should work on every Linux systems featuring basic tools.

  #!/bin/sh
  
  for pid in $@; do
  	cd /proc/"$pid"/fd || continue
  	
  	for fd in *; do
  		file=$(readlink "$fd");
  		if [ ! -z "$file" ]; then
  			size=$( (du -b "$file" 2>/dev/null || echo 0) | awk '{print $1}')
  			if [ "$size" = "0" ]; then
  				continue
  			fi
  			
  			pos=$(< ../fdinfo/"$fd" grep -F 'pos:' | grep -oP '\d+')
  			
  			percent=$(echo "100*${pos}/${size}" | bc -l | xargs printf "%2.2f")
  			echo "${file}: ${percent}% (${pos}/${size})"
  		fi
  	done
  done

EDIT: as mentioned elsewhere in the thread, the pv program has a way to watch a PID as well that makes this program unnecessary.

Not ugly at all!

    $ cp Test Test2
    ^T
    load: 3.42  cmd: cp 86526 running 0.01u 4.36s
    Test -> Test2   6%

I'd also like to map ^T to `kill -USR1 <pid>` on my machines, but didn't research it much.

> Sending an ‘INFO’ signal (or ‘USR1’ signal where that is unavailable) to a running dd process makes it print I/O statistics to standard error and then resume copying. [0]

[0] https://www.gnu.org/software/coreutils/manual/html_node/dd-i...

https://linux.die.net/man/1/pv

cp

    pv source > target

    tar czf images.tar.gz image1 image2 image3 | pv > target

    pv source | gzip > target.gz
    # or
    gzip < source | pv | target.gz

           -d PID[:FD], --watchfd PID[:FD]
              Instead of transferring data, watch file descriptor FD  of  process  PID,
              and  show  its progress.  The pv process will exit when FD either changes
              to a different file, changes read/write mode, or is  closed;  other  data
              transfer  modifiers  -  and  remote  control  - may not be used with this
              option.

              If only a PID is specified, then that process will be  watched,  and  all
              regular  files  and  block devices it opens will be shown with a progress
              bar.  The pv process will exit when process PID exits.

https://youtu.be/ui4DFIfqH_U

'man pv' is a short read and is something anyone who want to monitor such things should read.

gzip < source | pv > target.gz

It prints one period (.) for every 1000 lines of text piped into it. It just shows that there is still activity, not progress, but often this is enough.

A typical usage is:

  tar xvfz some_huge_tarball.tar.gz | dots

  #!/usr/bin/perl

  $| = 1;
  $i = 0;

  if ($#ARGV < 0) {
    $number = 1000;
  } else {
    $number = shift(@ARGV);
  }

  while (<STDIN>) {
    print '.' unless ($i % $number);
    $i++;
  }

  print "\n";

[1] http://man7.org/linux/man-pages/man1/watch.1.html

A Python version of the Linux watch command:

https://jugad2.blogspot.in/2018/05/a-python-version-of-linux...

plenty of our headless jobs know exactly where they are and how big the job is. There should be a standard dash for getting this.

even things like hadoop & luigi that have dashboards built in aren't great at progress & ETA

we should surface & store stats from any long-running loop. Further benefit that you can use prev runtime to properly schedule the next run (or run more sophisticated feedback & alerting).

A Unix Utility You Should Know About: Pipe Viewer:

http://www.catonmat.net/blog/unix-utilities-pipe-viewer/