Difference between revisions of "Linux command line"

From ScientificComputing
Jump to: navigation, search
m (→‎rm)
Line 278: Line 278:
 
  total 1
 
  total 1
 
  -rw-r--r-- 1 leonhard T0000 2775040 Jun 15 15:48 test.tar''''
 
  -rw-r--r-- 1 leonhard T0000 2775040 Jun 15 15:48 test.tar''''
 +
 +
===top===
 +
*Dynamic real-time view of of processes on a running system
 +
 +
===htop===
 +
*Interactive process viewer on a running system
  
 
==External links==
 
==External links==

Revision as of 07:54, 25 October 2016

Bash is a powerful scripting language, which can directly been used in a Linux terminal and is often combined with Linux commands. It can simplify the many tasks when you are working on a HPC cluster. Therefore we recommend to study at least some of the basic Linux and bash commands, before you start using the clusters. Below we give examples for bash and the most widely used Linux commands.

Bash

  • specify a variable and print its value to the standard output
[leonhard@euler01 ~]$ VAR_A="some string variable"
[leonhard@euler01 ~]$ echo $VAR_A
some string variable
  • Print numbers 1-5 to standard output with a for loop
[leonhard@euler01 ~]$ for i in {1..5}; do echo $i; done
1
2
3
4
5
  • Print all numbers with two digits to standard output without using a for loop
[leonhard@euler01 ~]$ echo {0..9}{0..9}                                                                
00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33
34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67
68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
  • Print 5 random number to standard output and index them
[leonhard@euler01 test]$ for i in {1..5}; do echo "$i-th random number $RANDOM"; done
1-th random number 25702
2-th random number 21605
3-th random number 6648
4-th random number 13259
5-th random number 8814
  • Do manipulations on a string (for instance exchange "home" by "work" in "/cluster/home/leonhard")
[leonhard@euler01 ~]$ STRING1="/cluster/home/leonhard"
[leonhard@euler01 ~]$ echo ${STRING1/home/work}
/cluster/work/leonhard


Linux commands

ls

  • Display list of files and folders sorted by time in reversed order (newest files at the bottom), including hidden files.
[leonhard@euler01 test]$ ls -ltra
total 1619
-rw-r--r--  1 leonhard T0000     166 Jun 13 17:12 file2
-rw-r--r--  1 leonhard T0000   26096 Jun 13 17:13 .file3
-rw-r--r--  1 leonhard T0000   44642 Jun 13 17:14 file4
-rw-r--r--  1 leonhard T0000 1488156 Jun 13 17:14 file1
-rw-r--r--  1 leonhard T0000      73 Jun 13 17:15 file5
drwx------ 71 leonhard T0000     117 Jun 13 17:15 ..
drwxr-xr-x  2 leonhard T0000       7 Jun 13 17:15 .
  • Display list of files sorted by filesize in reversed order (largest files at the bottom), including hidden files.
[leonhard@euler01 test]$ ls -lSra
total 1619
drwxr-xr-x  2 leonhard T0000       7 Jun 13 17:15 .
-rw-r--r--  1 leonhard T0000      73 Jun 13 17:15 file5
drwx------ 71 leonhard T0000     117 Jun 13 17:15 ..
-rw-r--r--  1 leonhard T0000     166 Jun 13 17:12 file2
-rw-r--r--  1 leonhard T0000   26096 Jun 13 17:13 .file3
-rw-r--r--  1 leonhard T0000   44642 Jun 13 17:14 file4
-rw-r--r--  1 leonhard T0000 1488156 Jun 13 17:14 file1

cd

  • Change to your home directory.
cd ~
  • Go one level up in the directory hierarchy.
cd ..
  • Change to a directory with path: /path/to/directory.
cd /path/to/directory

pwd

  • Print working directory.
[leonhard@euler01 test]$ pwd
/cluster/home/leonhard/test

echo

  • Print a string to the standard output.
[leonhard@euler01 test]$ echo "this goes to the standard output"
this goes to the standard output

less, cat, more

  • Display the content of a file.
less file

or 

cat file

or 

more file

cp

  • Copy a file.
[leonhard@euler01 test]$ ls -l
total 1540
-rw-r--r-- 1 leonhard T0000 1488156 Jun 13 17:14 file1
[leonhard@euler01 test]$ cp file1 file2
[leonhard@euler01 test]$ ls -l
total 1540
-rw-r--r-- 1 leonhard T0000 1488156 Jun 13 17:14 file1
-rw-r--r-- 1 leonhard T0000 1488156 Jun 13 17:26 file2
  • Copy a directory.
[leonhard@euler01 test]$ ls -l
total 1
drwxr-xr-x 2 leonhard T0000 2 Jun 13 17:26 testdir1
[leonhard@euler01 test]$ cp -r testdir1 testdir2
[leonhard@euler01 test]$ ls -l
total 2
drwxr-xr-x 2 leonhard T0000 2 Jun 13 17:26 testdir1
drwxr-xr-x 2 leonhard T0000 2 Jun 13 17:27 testdir2
[leonhard@euler01 test]$

mv

  • Rename a file.
[leonhard@euler01 test]$ ls -l
total 1
-rw-r--r-- 1 leonhard T0000 10 Jun 13 17:28 file1
[leonhard@euler01 test]$ mv file1 file2
[leonhard@euler01 test]$ ls -l
total 1
-rw-r--r-- 1 leonhard T0000 10 Jun 13 17:28 file2

rm

  • Delete a file.
[leonhard@euler01 test]$ ls -l
total 1
-rw-r--r-- 1 leonhard T0000 10 Jun 13 17:28 file2
[leonhard@euler01 test]$ rm file2
[leonhard@euler01 test]$ ls -l
total 0
  • Delete a directory (no matter if it contains files or not).
[leonhard@euler01 test]$ ls -l
total 1
drwxr-xr-x 2 leonhard T0000 2 Jun 13 17:30 testdir1
[leonhard@euler01 test]$ rm -r testdir1
[leonhard@euler01 test]$ ls -l
total 0

mkdir

  • Create a directory.
[leonhard@euler01 test]$ ls -l
total 0
[leonhard@euler01 test]$ mkdir testdir1
[leonhard@euler01 test]$ ls -l
total 1
drwxr-xr-x 2 leonhard T0000 2 Jun 13 17:31 testdir1

rmdir

  • Delete a directory (deletes only empty directories).
[leonhard@euler01 test]$ ls -l
total 1
drwxr-xr-x 2 sfux sdid 2 Jun 13 17:31 testdir1
[leonhard@euler01 test]$ rmdir testdir1
[leonhard@euler01 test]$ ls -l
total 0

man

  • Display the manual of a Linux command (output is too large to display it here, but just try it out).
man ls
man screen

exit

  • Terminate and exit the current shell.
[leonhard@euler01 ~]$ exit
logout
Connection to euler.ethz.ch closed.
leonhard@calculus:~$

vi, nano

  • Try out the text editors by displaying a file.
vi file

or 

nano file

grep

  • Search for a string within another string or a file.
[leonhard@euler01 test]$ ls -l
total 2
-rw-r--r-- 1 leonhard T0000 0 Jun 13 17:44 file11
-rw-r--r-- 1 leonhard T0000 0 Jun 13 17:44 file20
-rw-r--r-- 1 leonhard T0000 0 Jun 13 17:43 testfile1
-rw-r--r-- 1 leonhard T0000 0 Jun 13 17:43 testfile2
[leonhard@euler01 test]$ ls -l | grep test
-rw-r--r-- 1 leonhard T0000 0 Jun 13 17:43 testfile1
-rw-r--r-- 1 leonhard T0000 0 Jun 13 17:43 testfile2

sort

  • Sort a file line by line
[leonhard@euler01 test]$ more file1
c line c
b line b
d line d
a line a
[leonhard@euler01 test]$ sort file1
a line a
b line b
c line c
d line d

sed

  • Powerful tool for string manipulation (even whole files) which has many more functions. Exchange part of a string by another one (see also bash examples)
[leonhard@euler01 test]$ echo what a beautyful day | sed s/day/night/
what a beautyful night
[leonhard@euler01 test]$

awk

  • Awk is really a programming language. It can process text, perform arithmetic operations, etc. It is usually used in combination with another command to process/extract information. For example, to find out how many jobs are pending/running on the cluster:
[leonhard@euler01 ~]$ bqueues
QUEUE_NAME      PRIO STATUS          MAX JL/U JL/P JL/H NJOBS  PEND   RUN  SUSP 
clc              94  Open:Active     256    -    -    -     0     0     0     0
bigmem.4h        88  Open:Active       -    -    -    -   264   240    24     0
bigmem.24h       86  Open:Active       -    -    -    -  3031  2550   481     0
bigmem.120h      84  Open:Active       -  960    -    -   834   416   418     0
normal.4h        68  Open:Active       -    -    -    -  7520  5431  2089     0
normal.24h       66  Open:Active       -    -    -    - 72830 66192  6638     0
normal.120h      64  Open:Active       -    -    -    - 13666  5251  8415     0
normal.30d       62  Open:Active       -    -    -    -    58    58     0     0
virtual.40d      58  Open:Active       -    -    -    -     0     0     0     0
light.5d         10  Open:Active       -    -    -    -     1     0     1     0
purgatory         1  Open:Inact        -    -    -    -     4     4     0     0

[leonhard@euler01 ~]$ bqueues |  awk 'BEGIN{p=0;r=0} NR>1{p+=$9;r+=$10} END{printf "Pending:%8i\nRunning:%8i\n",p,r}'
Pending:   80142
Running:   18069

find

Search a string (substitute 'searchstring' by the string you are searching for) in all files contained in the current directory and all its subdirectories. 

find . -type f -exec grep -q 'searchstring' {} \; -a -exec ls -l {} \;

du

  • Go to a directory and display the total size of it and then the sizes of all subdirectories.
[leonhard@euler01 mathematica]$ du -hs
8.6M    .
[leonhard@euler01 mathematica]$ du -h
17K     ./parallel
5.0K    ./batch_mode
24K     ./plot_graphics
1.3M    ./ptest
7.3M    ./ptest2
8.6M    .

tar

  • Create a tar archive including several files.
[leonhard@euler01 test]$ ls
PtCO.t21  t21.C  t21.O  t21.Pt
[leonhard@euler01 test]$ tar -cf test.tar *
[leonhard@euler01 test]$ ls -ltr
total 5903
-rw------- 1 leonhard T0000  675840 May 23 15:27 t21.Pt
-rw------- 1 leonhard T0000  442368 May 23 15:27 t21.O
-rw------- 1 leonhard T0000  442368 May 23 15:27 t21.C
-rw------- 1 leonhard T0000 1204224 May 23 15:27 PtCO.t21
-rw-r--r-- 1 leonhard T0000 2775040 Jun 15 15:48 test.tar

gzip

  • Compress a file or an archive (tar and gzip are often used together) using gzip.
[leonhard@euler01 test]$ ls -ltr
total 2821
-rw-r--r-- 1 leonhard T0000 2775040 Jun 15 15:48 test.tar
[leonhard@euler01 test]$ gzip test.tar
[leonhard@euler01 test]$ ls -ltr
total 1
-rw-r--r-- 1 leonhard T0000 730947 Jun 15 15:48 test.tar.gz
[leonhard@euler01 test]$ gunzip test.tar.gz 
[leonhard@euler01 test]$ ls -ltr
total 1
-rw-r--r-- 1 leonhard T0000 2775040 Jun 15 15:48 test.tar

bzip2

  • Compress a file or an archive using bzip2
[leonhard@euler01 test]$ ls -ltr
total 2820
-rw-r--r-- 1 leonhard T0000 2775040 Jun 15 15:48 test.tar
[leonhard@euler01 test]$ bzip2 test.tar 
[leonhard@euler01 test]$ ls -ltr
total 1
-rw-r--r-- 1 leonhard T0000 638158 Jun 15 15:48 test.tar.bz2
[leonhard@euler01 test]$ bunzip2 test.tar.bz2 
[leonhard@euler01 test]$ ls -ltr
total 1
-rw-r--r-- 1 leonhard T0000 2775040 Jun 15 15:48 test.tar'

top

  • Dynamic real-time view of of processes on a running system

htop

  • Interactive process viewer on a running system

External links

Retrieved from "https://scicomp.ethz.ch/w/index.php?title=Linux_command_line&oldid=1252"