In centos 7, what command is to to extract fields from a file line (record)?

To search for color at the end of the line we could use the character $, designating the end of line marker, note that we do not need the extended regular expressions with this search

$ which mkfifo
/usr/bin/mkfifo

Reversing this a little we could use the carat, ^ , to search for lines beginning with :

$ which mkfifo
/usr/bin/mkfifo

Again we can use the word boundary with an extended search to exclude the additional line:

$ which mkfifo
/usr/bin/mkfifo

But what if we want both color and colour? US and UK spellings. We can make the u optional bu using the ? meta-character in the regular expression. The ? applies to the previous character making it optional:

$ which mkfifo
/usr/bin/mkfifo

Should we want to search for ranges on characters in the regular expression we can use square brackets. If we want to search for lines that begin with n or N then we could use the -i option with grep for a case insensitive search; alternatively:

$ grep ^[nN] test.txt
no color

But be careful, a misplaced ^ can easily reverse the search. A carat inside the brackets indicates that we are not looking for lines that start with n or N. We could also use the -v option to grep in invert the complete search. In the example note that we use the ^ outside the brackets as before denoting the lines starts with and then the carat inside that denotes not n or N. So the line must start with anything other than n or N.

$ which mkfifo
/usr/bin/mkfifo

Searching for Files Using Find

If you have not already found the command find (/usr/bin/find) then you will need to find it soon. We can use find in a similar way to ls. If used on its own, just the word find, find will list all files in the current directory and below. The behavior of find is to recurse automatically, listing subdirectory content. The output can be extensive especially if run higher up in the file-system. So we can run find with options to set criteria for the search, we can also control the recursion, limiting the depth of directories searched.

$ which mkfifo
/usr/bin/mkfifo

The command above will list only directories , (-type d ), within the current directory and below

$ which mkfifo
/usr/bin/mkfifo

The find syntax above with search the /var directory for directories, the maxdepth option limits the search to this directory only, 1 level down. The additional criteria searches for the file permissions including the special group permissions, These two criteria are ANDed together; we could use -o to OR them together. We are then looking for directories which have the SGID bit set that are directly below the /var directory.

The criteria that we can use in are searches include:

• -type : file type being a value of:
  • f for regular files
  • l for symbolic links
  • d for directories
  • c for character devices
  • b for block devices
  • p for pipes
  • s for sockets
• -perm | files with certain permissions
• -atime | last accessed time
• -mtime | last modified time
• -size | file size
• -inum | find files based on the inode number
• And many more. The man page for find is very good with lots of examples

Find also has actions, the default action for find is to print to the screen. It is optionals but the following two commands are the same, displaying symbolic links from the /etc directory down:

$ which mkfifo
/usr/bin/mkfifo

Another simple action is -delete, you are not prompted to delete files; but those files meeting the criteria are deleted:

$ find $HOME/Documents/ -type f -atime +365 -delete

The Documents directory in the current users home directory is searched for files that have not been accessed in the last 365 days.

Very powerfully we can use -exec or -ok to run any command against the found files. The exec action will proceed without any prompts whereas the ok option will prompt for each file before any actions

$ which mkfifo
/usr/bin/mkfifo

The two commands above are similar: the first will prompt and the second will not. Both will run the action to delete (rm) the file name in the place holder {}. For each file located it is placed in the braces awaiting its imminent expunging. Any command can be used in place of rm, this is just an example. The next example removes the execute permissions only from files and not directories or links:

$ find $HOME/Documents/ -type f -exec chmod -x {}\ ;

Extracting Data Using Head or Tail

If we need to view the top of a file we can use head (/usr/bin/head) and should we need to view the end of a file we can use tail (/usr/bin/tail). The following command will list the first two lines of the file test.txt.

$ find / -type p 2>/dev/null
/run/dmeventd-client
/run/dmeventd-server
/run/systemd/inhibit/14.ref
/run/systemd/inhibit/13.ref
/run/systemd/sessions/211.ref
/run/systemd/initctl/fifo

Using the similar command tail we can display the last two lines:

$ find / -type p 2>/dev/null
/run/dmeventd-client
/run/dmeventd-server
/run/systemd/inhibit/14.ref
/run/systemd/inhibit/13.ref
/run/systemd/sessions/211.ref
/run/systemd/initctl/fifo

When reading log files it is common to follow the end of the log. This uses the -f option to tail and will continue to display the current last 10 lines of the log. Use control + c to stop following the file.

$ find / -type p 2>/dev/null
/run/dmeventd-client
/run/dmeventd-server
/run/systemd/inhibit/14.ref
/run/systemd/inhibit/13.ref
/run/systemd/sessions/211.ref
/run/systemd/initctl/fifo

Note: The log file used here is on Ubuntu on other systems the more general log file is /var/log/messages

Similarly we can use cat (/bin/cat) and tac (/usr/bin/tac), cat list or concatenates the file from top to bottom and tac from bottom to top. If you focus is on the bottom of the file use cat, you will be left at the bottom of the file. If your focus is on the top use tac as you will be left at the top of the file.

Counting Words, Lines or Characters Using wc

Using the command wc we are at the heart of extracting and searching data from files at the Linux command line. The command wc stands for word count but it can count much more that that. WE will step through some examples.

$ find / -type p 2>/dev/null
/run/dmeventd-client
/run/dmeventd-server
/run/systemd/inhibit/14.ref
/run/systemd/inhibit/13.ref
/run/systemd/sessions/211.ref
/run/systemd/initctl/fifo

The first counts the lines, words and characters. The second line counts just lines, then just words and then just characters. The output from ls -l is shown below note that it shows both the line count and the file name.

$ find / -type p 2>/dev/null
/run/dmeventd-client
/run/dmeventd-server
/run/systemd/inhibit/14.ref
/run/systemd/inhibit/13.ref
/run/systemd/sessions/211.ref
/run/systemd/initctl/fifo

Searching and Extracting Data Using Fields and the Command Cut

The command cut (/usr/bin/cut) can be useful where viewing every filed in a file is not required. We may only want to see certain fields. Even with the output of a command we can pipe the output to cut. Suppose we only need the line count from wc not the file name:

What command in the Windows 10 CLI will display the contents of a text file on the standard output device?

What tab in the Command Prompt properties box allows you to select the screen buffer and window sizes of the Command Prompt?

How to save command line output in text file?

What command is used to display information about the device?