Softpanorama

May the source be with you, but remember the KISS principle ;-)
Contents Bulletin Scripting in shell and Perl Network troubleshooting History Humor

Redhat Network Configuration

News

Linux Networking

Recommended Books

Recommended Links

Network Utilities

Configuring the Services Setting a Default Gateway in RHEL
Ethernet Protocol Linux ifconfig How to change IP address in RHEL ethtool Autonegotiation Traceroute ntop
NTP RHEL NTP configuration Troubleshooting NTP on Red Hat Linux RHEL handling of DST change Changing timezone in RHEL6 from the command line    
Setting a Default Gateway in RHE NetworkManager Disabling RHEL 6 Network Manager        
Linux Routing Linux route command netstat Samba VNC on Linux SSH FTP
Sendmail on RHEL Postfix Nmap Horror Stories Unix History Humor Etc

The first step in configuring Linux TCP-IP stack is configuring the network card interface. You can configure network card by editing text files stored in /etc/sysconfig/network-scripts/ directory. First change directory to /etc/sysconfig/network-scripts/:

# cd /etc/sysconfig/network-scripts/

You need to edit /etc/sysconfig/network-scripts files as follows:

To set IP IP address and network mask: /sbin/ifconfig -a eth0 192.168.1.5 netmask 255.255.255.0

Verify the settings with /sbin/ifconfig eth0.

Add the default gateway: /sbin/route add default gw 192.168.1.254

Verify the gateway setting: /sbin/route. The line beginning with default should have your gateway under the gateway column.

Alternately, you can edit the file /etc/sysconfig/network-scripts/ifcfg-eth0 to look like (replace with your network numbers)

Broadcom Corporation NetXtreme BCM5761 Gigabit Ethernet PCIe
DEVICE=eth0
BOOTPROTO=none
BROADCAST=10.194.191.255
HWADDR=78:2B:CB:35:AB:C6
IPADDR=10.194.181.33
NETMASK=255.255.240.0
NETWORK=10.194.176.0
ONBOOT=yes
TYPE=Ethernet
USERCTL=no
IPV6INIT=no
PEERDNS=yes
GATEWAY=10.194.174.1

and the file /etc/sysconfig/network to look like (replace with your network numbers and hostname):

NETWORKING=yes
HOSTNAME=name.host.net
FORWARD_IPV4=yes
GATEWAYDEV=
GATEWAY=192.168.1.254

Please note that for static IP addressing you need to define either routes or default gateway and hostname in /etc/sysconfig/network file. For example if you do not plan to use IPv6, the setup can be something like:

NETWORKING=yes
NETWORKING_IPV6=no
HOSTNAME=yourhost
GATEWAY=192.168.1.254

To make changes you made in files active you need to restart networking using network RC script

/etc/init.d/network restart

You also need to define DNS in /etc/resolv.conf file

Ping the gateway and a few other computers on the network to verify your settings are correct before and after the reboot.

NFS

Try using NFSv3 if you are currently using NFSv2. There can be very significant performance increases with this change.

Increasing the read write block size. This is done with the rsize and wsize mount options. They need to the mount options used by the NFS clients. Values of 4096 and 8192 reportedly increase performance alot. But see the notes in the HOWTO about experimenting and measuring the performance implications. The limits on these are 8192 for NFSv2 and 32768 for NFSv3

Another approach is to increase the number of nfsd threads running. This is normally controlled by the nfsd init script. On Red Hat Linux machines, the value "RPCNFSDCOUNT" in the nfs init script controls this value. The best way to determine if you need this is to experiment. The HOWTO mentions a way to determin thread usage, but that doesn't seem supported in all kernels.

Another good tool for getting some handle on NFS server performance is `nfsstat`. This util reads the info in /proc/net/rpc/nfs[d] and displays it in a somewhat readable format. Some info intended for tuning Solaris, but useful for it's description of the nfsstat format

See also the tcp tuning info

Apache config

Make sure you starting a ton of initial daemons if you want good benchmark scores.

Something like:

#######
MinSpareServers 20
MaxSpareServers 80
StartServers 32

# this can be higher if apache is recompiled
MaxClients 256

MaxRequestsPerChild 10000
        
Note: Starting a massive amount of httpd processes is really a benchmark hack. In most real world cases, setting a high number for max servers, and a sane spare server setting will be more than adequate. It's just the instant on load that benchmarks typically generate that the StartServers helps with.

The MaxRequestPerChild should be bumped up if you are sure that your httpd processes do not leak memory. Setting this value to 0 will cause the processes to never reach a limit.

One of the best resources on tuning these values, especially for app servers, is the mod_perl performance tuning documentation.

Bumping the number of available httpd processes

The biggest scalability problem with apache, 1.3.x versions at least, is it's model of using one process per connection. In cases where there large amounts of concurent connections, this can require a large amount resources. These resources can include RAM, schedular slots, ability to grab locks, database connections, file descriptors, and others.

In cases where each connection takes a long time to complete, this is only compunded. Connections can be slow to complete because of large amounts of cpu or i/o usage in dynamic apps, large files being transfered, or just talking to clients on slow links.

There are several strategies to mitigate this. The basic idea being to free up heavyweight apache processes from having to handle slow to complete connections.

Static Content Servers

Samba Tuning

Depending on the type of tests, there are a number of tweaks you can do to samba to improve its performace over the default. The default is best for general purpose file sharing, but for extreme uses, there are a couple of tweaks.

The first one is to rebuild it with mmap support. In cases where you are serving up a large amount of small files, this seems to be particularly useful. You just need to add a "--with-mmap" to the configure line.

You also want to make sure the following options are enabled in the /etc/smb.conf file:

read raw = no
read prediction = true
level2 oplocks = true

One of the better resources for tuning samba is the "Using Samba" book from O'reily. The chapter on performance tuning is available online.

Openldap tuning

The most important tuning aspect for OpenLDAP is deciding what attributes you want to build indexes on.

I use the values:

cachesize 10000
dbcachesize 100000
sizelimit 10000
loglevel 0
dbcacheNoWsync

index cn,uid
index uidnumber
index gid
index gidnumber
index mail

If you add the following parameters to /etc/openldap/slapd.conf before entering the info into the database, they will all get indexed and performance will increase.

netstat

Since this document is primarily concerned with network servers, the `netstat` command can often be very useful. It can show status of all incoming and outgoing sockets, which can give very handy info about the status of a network server.

One of the more useful options is:

        netstat -pa

The `-p` options tells it to try to determine what program has the socket open, which is often very useful info. For example, someone nmap's their system and wants to know what is using port 666 for example. Running netstat -pa will show you its satand running on that tcp port.

One of the most twisted, but useful invocations is:

netstat -a -n|grep -E "^(tcp)"| cut -c 68-|sort|uniq -c|sort -n

This will show you a sorted list of how many sockets are in each connection state. For example:

      9  LISTEN      
     21  ESTABLISHED 

Top Visited
Switchboard
Latest
Past week
Past month

NEWS CONTENTS

Old News ;-)

[Aug 05, 2017] Disabling NetworkManager on RHEL 7

Aug 01, 2017 | superuser.com
Andrew 46

I was setting up a RHEL7 server in vmware vSphere and I'm having trouble getting it on the network without NetworkManager. I configured the server to have a static IP during the install process and it set everything up using NetworkManager. While this does work we do not use NetworkManager in my office, so I went and entered what we usually put the config file to get RHEL6 servers online without NetworkManager.

/etc/sysconfig/network-scripts/ifcfg-ens192 is the following:

NAME=ens192
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=no
BOOTPROTO=static
IPADDR=10.0.2.15
PREFIX=24
GATEWAY=10.0.2.2

However when I disable NetworkManager the network service fails to start with the following error

#service network restart

Restarting network (via systemctl): Job for network.service failed. See 'systemctl status network.service' and 'journalctl -xn' for details.

And both commands output the following:

network[1838]: RTNETLINK answers: File exists
network[1838]: RTNETLINK answers: File exists
network[1838]: RTNETLINK answers: File exists
network[1838]: RTNETLINK answers: File exists
network[1838]: RTNETLINK answers: File exists
network[1838]: RTNETLINK answers: File exists
network[1838]: RTNETLINK answers: File exists
systemd[1]: network.service: control process exited, code=exited status=1
systemd[1]: Failed to start LSB: Bring up/down networking

Also, here's what the command 'ip addr' outputs:

1: lo: mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens192: mtu 1500 qdisc noop state DOWN qlen 1000
link/ether 08:00:27:98:8e:df brd ff:ff:ff:ff:ff:ff
asked Jul 11 '14 at 22:19

Pavel Šimerda Apr 5 '15 at 5:40

I recently debugged an issue with network.service and the best way to track the ip commands was strace . You shouldn't generally get this type of error. It might be worth reporting (ideally via support). –

nickg Oct 25 '14 at 16:46

Check your MAC Address for the VM. It should be 08:00:27:98:8e:df since that is what is shown you ran ip addr. If it's anything else, you will need to set it in your ifcfg-ens192 file with the following, but replace the address with the actual.

HWADDR="08:00:27:98:8e:df"

I had the same issue and this solved it for me.

Pavel Šimerda Apr 5 '15 at 5:43

The configuration file in the Question apparently relies on NAME=ens192 without any MAC address matching.

Dec 30 '14 at 14:46 0ldd0g

All I found that it takes to resolve this is that MAC in the Config
 NAME=ens192
 TYPE=Ethernet
 ONBOOT=yes
 HWADDR="08:00:27:98:8e:df"
 NM_CONTROLLED=no
 BOOTPROTO=static
 IPADDR=10.0.2.15
 PREFIX=24
 GATEWAY=10.0.2.2

If you are not sure of the hardware address you can find it in.

 cat /sys/class/net/ens192/address
you should put that information (GATEWAY=10.0.2.2) in /etc/sysconfig/network once it's done, restarting the service should succeed

Try to go to the virtual machine network settings and make sure the network cable is connected and check if you have blocked this with a firewall

NetworkManager dictates the default route ( ip route ) even though your interface has nm disabled, it is just that interface not the whole system.
ps aux | grep -I net   # will probably find NetworkManager still running.
chkconfig NetworkManager off
systemctl disable NetworkManager.service

systemctl disable doesn't stop a service, nor does chkconfig ... off which basically translates to the same command anyway. – Pavel Šimerda

I too came across "Failed to start LSB: Bring up/down networking" error, since disabling NetworkManager. It took two minutes to be brought up interfaces after boot. The cause of confusion was "... LSB". It turned out the message comes out from just the traditional /etc/rc.d/init.d/network script. In my case, following solved the problem;

To network-scripts/ifcfg-eth0 added

NMCONTROLLED=no

Removed unnecessary ifcfg-* files which NetworkManager has left behind

# rm /etc/sysconfig/network-scripts/ifcfg-Wired_connection_?
Abdullah  May 11 at 10:26
This will solve the problem!
# rm /etc/udev/rules.d/70-persistent-ipoib.rules 

# reboot

-Restart the networking service

 #systemctl restart network.service

NOW! Working.

I was having the same issue. So I just delete the backup files I made in /etc/sysconfig/network-scripts , such as ifcfg-Bridge_connection_1.home and ifcfg-Bridge_connection_1.office which I created for backup usage. They should not be created there. The /etc/init.d/network restart could work well after delete those useless ifcfg-*.

[Aug 04, 2017] Disabling Network Manager

Aug 04, 2017 | access.redhat.com
OpenStack networking currently does not work on systems that have the Network Manager ( NetworkManager ) service enabled. The Network Manager service is currently enabled by default on Red Hat Enterprise Linux installations where one of these package groups was selected during installation: The Network Manager service is not currently enabled by default on Red Hat Enterprise Linux installations where one of these package groups was selected during installation: Follow the steps listed in this procedure while logged in as the root user on each system in the environment that will handle network traffic. This includes the system that will host the OpenStack Networking service, all network nodes, and all compute nodes. These steps ensure that the NetworkManager service is disabled and replaced by the standard network service for all interfaces that will be used by OpenStack Networking.
  1. Verify Network Manager is currently enabled using the chkconfig command.
    # chkconfig --list NetworkManager
    The output displayed by the chkconfig command inicates whether or not the Network Manager service is enabled.
    • The system displays an error if the Network Manager service is not currently installed:
      error reading information on service NetworkManager: No such file or directory
      
      If this error is displayed then no further action is required to disable the Network Manager service.
    • The system displays a list of numerical run levels along with a value of on or off indicating whether the Network Manager service is enabled when the system is operating in the given run level.
      NetworkManager     0:off   1:off   2:off   3:off   4:off   5:off   6:off
      
      If the value displayed for all run levels is off then the Network Manager service is disabled and no further action is required. If the value displayed for any of the run levels is on then the Network Manager service is enabled and further action is required.
  2. Ensure that the Network Manager service is stopped using the service command.
    # service NetworkManager stop
    
  3. Ensure that the Network Manager service is disabled using the chkconfig command.
    # chkconfig NetworkManager off
    
  4. Open each interface configuration file on the system in a text editor. Interface configuration files are found in the /etc/sysconfig/network-scripts/ directory and have names of the form ifcfg- X where X is replaced by the name of the interface. Valid interface names include eth0 , p1p5 , and em1 . In each file ensure that the NM_CONTROLLED configuration key is set to no and the ON_BOOT configuration key is set to yes .
    NM_CONTROLLED=no
    ONBOOT=yes
    
    This action ensures that the standard network service will take control of the interfaces and automatically activate them on boot.
  5. Ensure that the network service is started using the service command.
    # service network start
  6. Ensure that the network service is enabled using the chkconfig command.
    # chkconfig network on

The Network Manager service has been disabled. The standard network service has been enabled and configured to control the required network interfaces.

[Aug 02, 2017] Issue with RHEL7 and disabling NetworkManager

Aug 02, 2017 | serverfault.com

cjmaio Jul 2 '14 at 15:14

Okay community, let's see if we can figure this one out, cause I'm out of answers.

Where I work I am setting up a bunch of RedHat Enterprise Linux servers. There is a collection of RHEL6 and RHEL7 servers.


On the RHEL6 servers, I am using the standard network configuration tool by configuring it in /etc/sysconfig/network-scripts/ifcfg-eth0 and a dhclient configuration file in /etc/dhclient-eth0.conf . Everything works properly, I am assigned the custom FQDN by our DNS servers (e.g. hostname.ad.company.tld ) and when the DHCP lease is up, it is renewed automatically.


Here is the issue:

In RHEL7, NetworkManager is enabled by default. On our Kickstart, I have removed NetworkManager and went back to configuring network and dhcp the way it is done in RHEL6. All of the configuration is the same (sans using /etc/sysconfig/network-scripts/ifcfg-ens192 instead of eth0) and works fine for the first DHCP lease.

Once the lease is up, it seemingly doesn't renew it until I issue a systemctl restart network command.


I have looked and looked and I am coming up short. There must be something different in RHEL7 or something not configured when you disable NetworkManager , but I cannot for the life of me figure it out.

Anyone have any thoughts?

As I know these usually help, I'll post my RHEL7 configuration files, and the snippet from the logs where it loses the DHCP lease.


/etc/sysconfig/network-scripts/ifcfg-ens192

# Generated by dracut initrd
DEVICE="ens192"
ONBOOT=yes
NETBOOT=yes
UUID="c23045ff-7b60-4dff-b052-30a61923a852"
IPV6INIT=yes
BOOTPROTO=dhcp
HWADDR="00:0c:29:b6:d8:cc"
TYPE=Ethernet
NAME="ens192"
NM_CONTROLLED=no

/etc/dhclient-ens192.conf

send host-name "hostname";
send fqdn.fqdn "hostname.ad.company.tld";
send fqdn.server-update off;

/var/log/messages

Jun 27 23:06:09 sa-kbwiki01 avahi-daemon[591]: Withdrawing address record for 129.89.78.221 on ens192.
Jun 27 23:06:09 sa-kbwiki01 avahi-daemon[591]: Leaving mDNS multicast group on interface ens192.IPv4 with address xxx.xx.xx.xxx.
Jun 27 23:06:09 sa-kbwiki01 avahi-daemon[591]: Interface ens192.IPv4 no longer relevant for mDNS.

That log snippet doesn't show your DHCP lease being lost. Keep looking, there should be other more relevant entries. – Michael Hampton ♦ Jul 2 '14 at 15:24

From what I recall hearing pre-launch is that networkManager is not the same PoS it was years ago and Red Hat more or less forces you to learn to live with it. Having said that, the documentation mentions that NetworkManager has been made responsible for starting dhclient, so it could be that without NM, dhclient is run with the -1 option and doesn't become a daemon. – HBruijn ♦ Jul 2 '14 at 15:36

@MichaelHampton I do not see anything else in /var/log/messages. Other things that use the network are operating fine until that line, at which point everything starts saying no network available. – cjmaio Jul 2 '14 at 15:59

@HBruijn That gives me somewhere to start... though when doing a ps aux | grep dhclient I do see that the -1 flag is being set... is there anywhere else that dhclient would log to other than /var/log/messages ? – cjmaio Jul 2 '14 at 16:00

Yeah, NM is fairly safe to use these days unless you have a very complicated setup. I do wonder why you're running Avahi though. – Michael Hampton ♦ Jul 2 '14 at 16:01

[Aug 26, 2015] Changing timezone in RHEL6 from the command line

[Apr 29, 2011] Disabling avahi-daemon Len

One of the things I quickly found to be bothering me is the fact that there was an apparently long and unexplicable delay for all new network connections which resembled to a dns resolving. No reason for lengthy dns resolving though. So I did a strace:
socket(PF_FILE, SOCK_STREAM, 0)         = 4
fcntl64(4, F_GETFD)                     = 0
fcntl64(4, F_SETFD, FD_CLOEXEC)         = 0
connect(4, {sa_family=AF_FILE, path="/var/run/avahi-daemon/socket"}, 110) = 0
fcntl64(4, F_GETFL)                     = 0x2 (flags O_RDWR)
fstat64(4, {st_mode=S_IFSOCK|0777, st_size=0, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7f35000_
llseek(4, 0, 0xbfa7d918, SEEK_CUR)     = -1 ESPIPE (Illegal seek)
write(4, "RESOLVE-ADDRESS 10.0.0.6\n", 25) = 25read(4,  <unfinished ...>

the results shows a connection to a avahi-daemon which I have no ideea what is good for so I should not need it. I disabled it in /etc/default/avahi-daemon

cat /etc/default/avahi-daemon # 0 = don't start, 1 = start
AVAHI_DAEMON_START=0

Hope it helps.

[Jul 25, 2010] Configure The Network Card In Redhat 5.3 Advanced Server

Edit The Network Configuration Files Directly

From a terminal window type:

cd /etc/sysconfig/networking-scripts

You will need to edit the following filess:

/etc/sysconfig/network-scripts/ifcfg-eth0  (and eth1 if you have another card.)
/etc/sysconfig/network-scripts/ifcfg-eth0

Change or Add New Info

cd /etc/sysconfig/network-scripts/
vi ifcfg-eth0

Change or Add The Following Lines

# Xen Virtual Ethernet
DEVICE=eth0
BOOTPROTO=none
BROADCAST=10.10.1.255
HWADDR=56:36:DC:8F:D5:59
IPADDR=10.10.1.73
NETMASK=255.255.255.0
NETWORK=10.10.1.0
ONBOOT=yes
GATEWAY=10.10.1.1
TYPE=Ethernet
 

Setup A Default Gateway

vi /etc/sysconfig/network

NETWORKING=yes
NETWORKING_IPV6=no
HOSTNAME=myhostname
GATEWAY=10.10.1.1

Make sure your DNS entries are correct. Set them to the correct values, whatever those are. For example:

vi /etc/resolv.conf

nameserver 10.10.1.13
nameserver 10.10.1.14
search mydomain.com

Save the file & restart the network service:

service network restart
The GUI Tool

You can also launch the system-config-network tool in GUI mode. From a command line where you are running X-Windows, type system-config-network, or chose System / Administration / Network from the menu

Linux Network Administrator's Guide, 2nd Edition Chapter 5 Configuring TCP-IP Networking

A couple of commands are used to configure the network interfaces and initialize the routing table. These tasks are usually performed from the network initialization script each time you boot the system. The basic tools for this process are called ifconfig (where "if" stands for interface) and route.

ifconfig is used to make an interface accessible to the kernel networking layer. This involves the assignment of an IP address and other parameters, and activation of the interface, also known as "bringing up" the interface. Being active here means that the kernel will send and receive IP datagrams through the interface. The simplest way to invoke it is with:

ifconfig interface ip-address

This command assigns ip-address to interface and activates it. All other parameters are set to default values. For instance, the default network mask is derived from the network class of the IP address, such as 255.255.0.0 for a class B address. ifconfig is described in detail in the section "All About ifconfig".

route allows you to add or remove routes from the kernel routing table. It can be invoked as:

route [add|del] [-net|-host] target [if]

The add and del arguments determine whether to add or delete the route to target. The -net and -host arguments tell the route command whether the target is a network or a host (a host is assumed if you don't specify). The if argument is again optional, and allows you to specify to which network interface the route should be directed -- the Linux kernel makes a sensible guess if you don't supply this information. This topic will be explained in more detail in succeeding sections.

The Loopback Interface

The very first interface to be activated is the loopback interface:

# ifconfig lo 127.0.0.1

Occasionally, you will see the dummy hostname localhost being used instead of the IP address. ifconfig will look up the name in the hosts file, where an entry should declare it as the hostname for 127.0.0.1:

# Sample /etc/hosts entry for localhost
localhost     127.0.0.1

To view the configuration of an interface, you invoke ifconfig, giving it only the interface name as argument:

$ ifconfig lo
lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:3924  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          Collisions:0 

As you can see, the loopback interface has been assigned a netmask of 255.0.0.0, since 127.0.0.1 is a class A address.

Now you can almost start playing with your mini-network. What is still missing is an entry in the routing table that tells IP that it may use this interface as a route to destination 127.0.0.1. This is accomplished by using:

# route add 127.0.0.1

Again, you can use localhost instead of the IP address, provided you've entered it into your /etc/hosts.

Next, you should check that everything works fine, for example by using ping.

# ping localhost
PING localhost (127.0.0.1): 56 data bytes
64 bytes from 127.0.0.1: icmp_seq=0 ttl=255 time=0.4 ms
64 bytes from 127.0.0.1: icmp_seq=1 ttl=255 time=0.4 ms
64 bytes from 127.0.0.1: icmp_seq=2 ttl=255 time=0.4 ms
^C
--- localhost ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.4/0.4/0.4 ms
#

When you invoke ping as shown here, it will continue emitting packets forever, unless interrupted by the user. The ^C marks the place where we pressed Ctrl-C.

The previous example shows that packets for 127.0.0.1 are properly delivered and a reply is returned to ping almost instantaneously. This shows that you have successfully set up your first network interface.

If the output you get from ping does not resemble that shown in the previous example, you are in trouble. Check any errors if they indicate that some file hasn't been installed properly. Check that the ifconfig and route binaries you use are compatible with the kernel release you run, and above all, that the kernel has been compiled with networking enabled (you see this from the presence of the /proc/net directory). If you get an error message saying "Network unreachable," you probably got the route command wrong. Make sure you use the same address you gave to ifconfig.

The steps previously described are enough to use networking applications on a standalone host. After adding the lines mentioned earlier to your network initialization script and making sure it will be executed at boot time, you may reboot your machine and try out various applications. For instance, telnet localhost should establish a telnet connection to your host, giving you a login: prompt.

However, the loopback interface is useful not only as an example in networking books, or as a test bed during development, but is actually used by some applications during normal operation.[5] Therefore, you always have to configure it, regardless of whether your machine is attached to a network or not.

[5] For example, all applications based on RPC use the loopback interface to register themselves with the portmapper daemon at startup. These applications include NIS and NFS.

Ethernet Interfaces

Configuring an Ethernet interface is pretty much the same as the loopback interface; it just requires a few more parameters when you are using subnetting.

At the Virtual Brewery, we have subnetted the IP network, which was originally a class B network, into class C subnetworks. To make the interface recognize this, the ifconfig incantation would look like this:

# ifconfig eth0 vstout netmask 255.255.255.0

This command assigns the eth0 interface the IP address of vstout (172.16.1.2). If we omitted the netmask, ifconfig would deduce the netmask from the IP network class, which would result in an incorrect netmask of 255.255.0.0. Now a quick check shows:

# ifconfig eth0
eth0      Link encap 10Mps Ethernet HWaddr  00:00:C0:90:B3:42
          inet addr 172.16.1.2 Bcast 172.16.1.255 Mask 255.255.255.0
          UP BROADCAST RUNNING  MTU 1500  Metric 1
          RX packets 0 errors 0 dropped 0 overrun 0
          TX packets 0 errors 0 dropped 0 overrun 0

You can see that ifconfig automatically sets the broadcast address (the Bcast field) to the usual value, which is the host's network number with all the host bits set. Also, the maximum transmission unit (the maximum size of IP datagrams the kernel will generate for this interface) has been set to the maximum size of Ethernet packets: 1,500 bytes. The defaults are usually what you will use, but all these values can be overidden if required, with special options that will be described under "All About ifconfig".

Just as for the loopback interface, you now have to install a routing entry that informs the kernel about the network that can be reached through eth0. For the Virtual Brewery, you might invoke route as:

# route add -net 172.16.1.0

At first this looks a little like magic, because it's not really clear how route detects which interface to route through. However, the trick is rather simple: the kernel checks all interfaces that have been configured so far and compares the destination address (172.16.1.0 in this case) to the network part of the interface address (that is, the bitwise AND of the interface address and the netmask). The only interface that matches is eth0.

Now, what's that -net option for? This is used because route can handle both routes to networks and routes to single hosts (as you saw before with localhost). When given an address in dotted quad notation, route attempts to guess whether it is a network or a hostname by looking at the host part bits. If the address's host part is zero, route assumes it denotes a network; otherwise, route takes it as a host address. Therefore, route would think that 172.16.1.0 is a host address rather than a network number, because it cannot know that we use subnetting. We have to tell route explicitly that it denotes a network, so we give it the -net flag.

Of course, the route command is a little tedious to type, and it's prone to spelling mistakes. A more convenient approach is to use the network names we defined in /etc/networks. This approach makes the command much more readable; even the -net flag can be omitted because route knows that 172.16.1.0 denotes a network:

# route add brew-net

Now that you've finished the basic configuration steps, we want to make sure that your Ethernet interface is indeed running happily. Choose a host from your Ethernet, for instance vlager, and type:

# ping vlager
PING vlager: 64 byte packets
64 bytes from 172.16.1.1: icmp_seq=0. time=11. ms
64 bytes from 172.16.1.1: icmp_seq=1. time=7. ms
64 bytes from 172.16.1.1: icmp_seq=2. time=12. ms
64 bytes from 172.16.1.1: icmp_seq=3. time=3. ms
^C
----vstout.vbrew.com PING Statistics----
4 packets transmitted, 4 packets received, 0
round-trip (ms)  min/avg/max = 3/8/12

If you don't see similar output, something is broken. If you encounter unusual packet loss rates, this hints at a hardware problem, like bad or missing terminators. If you don't receive any replies at all, you should check the interface configuration with netstat described later in "The netstat Command". The packet statistics displayed by ifconfig should tell you whether any packets have been sent out on the interface at all. If you have access to the remote host too, you should go over to that machine and check the interface statistics. This way you can determine exactly where the packets got dropped. In addition, you should display the routing information with route to see if both hosts have the correct routing entry. route prints out the complete kernel routing table when invoked without any arguments (-n just makes it print addresses as dotted quad instead of using the hostname):

# route -n
Kernel routing table
Destination  Gateway  Genmask         Flags Metric Ref Use    Iface
127.0.0.1    *        255.255.255.255 UH    1      0      112 lo
172.16.1.0   *        255.255.255.0   U     1      0       10 eth0

The detailed meaning of these fields is explained later in "The netstat Command". The Flags column contains a list of flags set for each interface. U is always set for active interfaces, and H says the destination address denotes a host. If the H flag is set for a route that you meant to be a network route, you have to reissue the route command with the -net option. To check whether a route you have entered is used at all, check to see if the Use field in the second to last column increases between two invocations of ping.

Routing Through a Gateway

In the previous section, we covered only the case of setting up a host on a single Ethernet. Quite frequently, however, one encounters networks connected to one another by gateways. These gateways may simply link two or more Ethernets, but may also provide a link to the outside world, such as the Internet. In order to use a gateway, you have to provide additional routing information to the networking layer.

The Ethernets of the Virtual Brewery and the Virtual Winery are linked through such a gateway, namely the host vlager. Assuming that vlager has already been configured, we just have to add another entry to vstout's routing table that tells the kernel it can reach all hosts on the Winery's network through vlager. The appropriate incantation of route is shown below; the gw keyword tells it that the next argument denotes a gateway:

# route add wine-net gw vlager

Of course, any host on the Winery network you wish to talk to must have a routing entry for the Brewery's network. Otherwise you would only be able to send data to the Winery network from the Brewery network, but the hosts on the Winery would be unable to reply.

This example describes only a gateway that switches packets between two isolated Ethernets. Now assume that vlager also has a connection to the Internet (say, through an additional SLIP link). Then we would want datagrams to any destination network other than the Brewery to be handed to vlager. This action can be accomplished by making it the default gateway for vstout:

# route add default gw vlager

The network name default is a shorthand for 0.0.0.0, which denotes the default route. The default route matches every destination and will be used if there is no more specific route that matches. You do not have to add this name to /etc/networks because it is built into route.

If you see high packet loss rates when pinging a host behind one or more gateways, this may hint at a very congested network. Packet loss is not so much due to technical deficiencies as to temporary excess loads on forwarding hosts, which makes them delay or even drop incoming datagrams.

Configuring a Gateway

Configuring a machine to switch packets between two Ethernets is pretty straightforward. Assume we're back at vlager, which is equipped with two Ethernet cards, each connected to one of the two networks. All you have to do is configure both interfaces separately, giving them their respective IP addresses and matching routes, and that's it.

It is quite useful to add information on the two interfaces to the hosts file as shown in the following example, so we have handy names for them, too:

172.16.1.1      vlager.vbrew.com    vlager vlager-if1
172.16.2.1      vlager-if2

The sequence of commands to set up the two interfaces is then:

# ifconfig eth0 vlager-if1
# route add brew-net
# ifconfig eth1 vlager-if2
# route add wine-net

If this sequence doesn't work, make sure your kernel has been compiled with support for IP forwarding enabled. One good way to do this is to ensure that the first number on the second line of /proc/net/snmp is set to 1.

IBM Redbooks Linux Performance and Tuning Guidelines

Abstract

Over the past few years, Linux has made its way into the data centers of many corporations all over the globe. The Linux operating system has become accepted by both the scientific and enterprise user population. Today, Linux is by far the most versatile operating system. You can find Linux on embedded devices such as firewalls and cell phones and mainframes. Naturally, performance of the Linux operating system has become a hot topic for both scientific and enterprise users. However, calculating a global weather forecast and hosting a database impose different requirements on the operating system. Linux has to accommodate all possible usage scenarios with the most optimal performance. The consequence of this challenge is that most Linux distributions contain general tuning parameters to accommodate all users.

IBMฎ has embraced Linux, and it is recognized as an operating system suitable for enterprise-level applications running on IBM systems. Most enterprise applications are now available on Linux, including file and print servers, database servers, Web servers, and collaboration and mail servers.

With use of Linux in an enterprise-class server comes the need to monitor performance and, when necessary, tune the server to remove bottlenecks that affect users. This IBM Redpaper describes the methods you can use to tune Linux, tools that you can use to monitor and analyze server performance, and key tuning parameters for specific server applications. The purpose of this redpaper is to understand, analyze, and tune the Linux operating system to yield superior performance for any type of application you plan to run on these systems.

The tuning parameters, benchmark results, and monitoring tools used in our test environment were executed on Red Hat and Novell SUSE Linux kernel 2.6 systems running on IBM System x servers and IBM System z servers. However, the information in this redpaper should be helpful for all Linux hardware platforms.

Update 4/2008: Typos corrected

[Feb 25, 2009] How to troubleshoot RHEL performance bottlenecks by Ken Milberg

09.30.2008

You've just had your first cup of coffee and have received that dreaded phone call. The system is slow. What are you going to do? This article will discuss performance bottlenecks and optimization in Red Hat Enterprise Linux (RHEL5).

Before getting into any monitoring or tuning specifics, you should always use some kind of tuning methodology. This is one which I've used successfully through the years:

1. Baseline – The first thing you must do is establish a baseline, which is a snapshot of how the system appears when it's performing well. This baseline should not only compile data, but also document your system's configuration (RAM, CPU and I/O). This is necessary because you need to know what a well-performing system looks like prior to fixing it.

2. Stress testing and monitoring – This is the part where you monitor and stress your systems at peak workloads. It's the monitoring which is key here – as you cannot effectively tune anything without some historic trending data.

3. Bottleneck identification – This is where you come up with the diagnosis for what is ailing your system. The primary objective of section 2 is to determine the bottleneck. I like to use several monitoring tools here. This allows me to cross-reference my data for accuracy.

4. Tune – Only after you've identified the bottleneck can you tune it.

5. Repeat – Once you've tuned it, you can start the cycle again – but this time start from step 2 (monitoring) – as you already have your baseline.

It's important to note that you should only make one change at a time. Otherwise, you'll never know exactly what impacted any changes which might have occurred. It is only by repeating your tests and consistently monitoring your systems that you can determine if your tuning is making an impact.

RHEL monitoring tools

Before we can begin to improve the performance of our system, we need to use the monitoring tools available to us to baseline. Here are some monitoring tools you should consider using:

Oprofile

This tool (made available in RHEL5) utilizes the processor to retrieve kernel system information about system executables. It allows one to collect samples of performance data every time a counter detects an interrupt. I like the tool also because it carries little overhead – which is very important because you don't want monitoring tools to be causing system bottlenecks. One important limitation is that the tool is very much geared towards finding problems with CPU limited processes. It does not identify processes which are sleeping or waiting on I/O.

The steps used to start up Oprofile include setting up the profiler, starting it and then dumping the data.

First we'll set up the profile. This option assumes that one wants to monitor the kernel.

# opcontrol --setup –vmlinux=/usr/lib/debug/lib/modules/'uname -r'/vmlinux

Then we can start it up.

# opcontrol --start

Finally, we'll dump the data.

# opcontrol --stop/--shutdown/--dump

SystemTap

This tool (introduced in RHEL5) collects data by analyzing the running kernel. It really helps one come up with a correct diagnosis of a performance problem and is tailor-made for developers. SystemTap eliminates the need for the developer to go through the recompile and reinstallation process to collect data.

Frysk

This is another tool which was introduced by Red Hat in RHEL5. What does it do for you? It allows both developers and system administrators to monitor running processes and threads. Frysk differs from Oprofile in that it uses 100% reliable information (similar to SystemTap) - not just a sampling of data. It also runs in user mode and does not require kernel modules or elevated privileges. Allowing one to stop or start running threads or processes is also a very useful feature.

Some more general Linux tools include top and vmstat. While these are considered more basic, often I find them much more useful than more complex tools. Certainly they are easier to use and can help provide information in a much quicker fashion.

Top provides a quick snapshot of what is going on in your system – in a friendly character-based display.

It also provides information on CPU, Memory and Swap Space.

Let's look at vmstat – one of the oldest but more important Unix/Linux tools ever created. Vmstat allows one to get a valuable snapshot of process, memory, sway I/O and overall CPU utilization.

Now let's define some of the fields:

Memory
swpd – The amount of virtual memory
free – The amount of free memory
buff – Amount of memory used for buffers
cache – Amount of memory used as page cache

Process
r – number of run-able processes
b – number or processes sleeping.
Make sure this number does not exceed the amount of run-able processes, because when this condition occurs it usually signifies that there are performance problems.

Swap
si – the amount of memory swapped in from disk
so – the amount of memory swapped out.

This is another important field you should be monitoring – if you are swapping out data, you will likely be having performance problems with virtual memory.

CPU
us – The % of time spent in user-level code.
It is preferable for you to have processes which spend more time in user code rather than system code. Time spent in system level code usually means that the process is tied up in the kernel rather than processing real data.
sy – the time spent in system level code
id – the amount of time the CPU is idle wa – The amount of time the system is spending waiting for I/O.

If your system is waiting on I/O – everything tends to come to a halt. I start to get worried when this is > 10.

There is also:

Free – This tool provides memory information, giving you data around the total amount of free and used physical and swap memory.

Now that we've analyzed our systems – lets look at what we can do to optimize and tune our systems.

CPU Overhead – Shutting Running Processes
Linux starts up all sorts of processes which are usually not required. This includes processes such as autofs, cups, xfs, nfslock and sendmail. As a general rule, shut down anything that isn't explicitly required. How do you do this? The best method is to use the chkconfig command.

Here's how we can shut these processes down.
[root ((Content component not found.)) _29_140_234 ~]# chkconfig --del xfs

You can also use the GUI - /usr/bin/system-config-services to shut down daemon process.

Tuning the kernel
To tune your kernel for optimal performance, start with:

sysctl – This is the command we use for changing kernel parameters. The parameters themselves are found in /proc/sys/kernel

Let's change some of the parameters. We'll start with the msgmax parameter. This parameter specifies the maximum allowable size of a single message in an IPC message queue. Let's view how it currently looks.

[root ((Content component not found.)) _29_139_52 ~]# sysctl kernel.msgmax
kernel.msgmax = 65536
[root ((Content component not found.)) _29_139_52 ~]#

There are three ways to make these kinds of kernel changes. One way is to change this using the echo command.

[root ((Content component not found.)) _29_139_52 ~]# echo 131072 >/proc/sys/kernel/msgmax
[root ((Content component not found.)) _29_139_52 ~]# sysctl kernel.msgmax
kernel.msgmax = 131072
[root ((Content component not found.)) _29_139_52 ~]#

Another parameter that is changed quite frequently is SHMMAX, which is used to define the maximum size (in bytes) for a shared memory segment. In Oracle this should be set large enough for the largest SGA size. Let's look at the default parameter:

# sysctl kernel.shmmax
kernel.shmmax = 268435456

This is in bytes – which translates to 256 MG. Let's change this to 512 MG, using the -w flag.

[root ((Content component not found.)) _29_139_52 ~]# sysctl -w kernel.shmmax=5368709132
kernel.shmmax = 5368709132
[root ((Content component not found.)) _29_139_52 ~]#

The final method for making changes is to use a text editor such as vi – directly editing the /etc/sysctl.conf file to manually make our changes.

To allow the parameter to take affect dynamically without a reboot, issue the sysctl command with the -p parameter.

Obviously, there is more to performance tuning and optimization than we can discuss in the context of this small article – entire books have been written on Linux performance tuning. For those of you first getting your hands dirty with tuning, I suggest you tread lightly and spend time working on development, test and/or sandbox environments prior to deploying any changes into production. Ensure that you monitor the effects of any changes that you make immediately; it's imperative to know the effect of your change. Be prepared for the possibility that fixing your bottleneck has created another one. This is actually not a bad thing in itself, as long as your overall performance has improved and you understand fully what is happening.

Performance monitoring and tuning is a dynamic process which does not stop after you have fixed a problem. All you've done is established a new baseline. Don't rest on your laurels, and understand that performance monitoring must be a routine part of your role as a systems administrator.

About the author: Ken Milberg is a systems consultant with two decades of experience working with Unix and Linux systems. He is a SearchEnterpriseLinux.com Ask the Experts advisor and columnist.

[Feb 23, 2009] Deployment_Guide/Gathering System Information

Before you learn how to configure your system, you should learn how to gather essential system> information. For example, you should know how to find the amount of free memory, the amount of available hard drive space, how your hard drive is partitioned, and what processes are running. This chapter discusses how to retrieve this type of information from your Red Hat Enterprise Linux system using simple commands and a few simple programs.

1. System Processes

The ps ax command displays a list of current system processes, including processes owned by other users. To display the owner alongside each process, use the ps aux command. This list is a static list; in other words, it is a snapshot of what was running when you invoked the command. If you want a constantly updated list of running processes, use top as described below. The ps output can be long. To prevent it from scrolling off the screen, you can pipe it through less:

ps aux | less

You can use the ps command in combination with the grep command to see if a process is running. For example, to determine if Emacs is running, use the following command:

ps ax | grep emacs

The top command displays currently running processes and important information about them including their memory and CPU usage. The list is both real-time and interactive. An example of output from the top command is provided as follows:

To exit top press the q key. Useful interactive commands that you can use:

For more information, refer to the
top(1) manual page.

Recommended Links

IBM Redbooks Linux Performance and Tuning Guidelines (June 05, 2007)

Over the past few years, Linux has made its way into the data centers of many corporations all over the globe. The Linux operating system has become accepted by both the scientific and enterprise user population. Today, Linux is by far the most versatile operating system. You can find Linux on embedded devices such as firewalls and cell phones and mainframes. Naturally, performance of the Linux operating system has become a hot topic for both scientific and enterprise users. However, calculating a global weather forecast and hosting a database impose different requirements on the operating system. Linux has to accommodate all possible usage scenarios with the most optimal performance. The consequence of this challenge is that most Linux distributions contain general tuning parameters to accommodate all users.

IBMฎ has embraced Linux, and it is recognized as an operating system suitable for enterprise-level applications running on IBM systems. Most enterprise applications are now available on Linux, including file and print servers, database servers, Web servers, and collaboration and mail servers.

With use of Linux in an enterprise-class server comes the need to monitor performance and, when necessary, tune the server to remove bottlenecks that affect users. This IBM Redpaper describes the methods you can use to tune Linux, tools that you can use to monitor and analyze server performance, and key tuning parameters for specific server applications. The purpose of this redpaper is to understand, analyze, and tune the Linux operating system to yield superior performance for any type of application you plan to run on these systems.

The tuning parameters, benchmark results, and monitoring tools used in our test environment were executed on Red Hat and Novell SUSE Linux kernel 2.6 systems running on IBM System x servers and IBM System z servers. However, the information in this redpaper should be helpful for all Linux hardware platforms. >

dkftpbench

http://www.kegel.com/

http://linuxperf.nl.linux.org/

http://www.citi.umich.edu/projects/citi-netscape/

NFS Performance Tunging

http://home.att.net/~jageorge/performance.html

http://www.linux.com/tuneup/

http://www.psc.edu/networking/perf_tune.html#Linux

Utilities



Etc

FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available in our efforts to advance understanding of environmental, political, human rights, economic, democracy, scientific, and social justice issues, etc. We believe this constitutes a 'fair use' of any such copyrighted material as provided for in section 107 of the US Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit exclusivly for research and educational purposes.   If you wish to use copyrighted material from this site for purposes of your own that go beyond 'fair use', you must obtain permission from the copyright owner. 

ABUSE: IPs or network segments from which we detect a stream of probes might be blocked for no less then 90 days. Multiple types of probes increase this period.  

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotesSomerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose BierceBernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOSProgramming Languages History : PL/1 : Simula 67 : C : History of GCC developmentScripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-MonthHow to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least


Copyright ฉ 1996-2016 by Dr. Nikolai Bezroukov. www.softpanorama.org was created as a service to the UN Sustainable Development Networking Programme (SDNP) in the author free time. This document is an industrial compilation designed and created exclusively for educational use and is distributed under the Softpanorama Content License.

The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Original materials copyright belong to respective owners. Quotes are made for educational purposes only in compliance with the fair use doctrine.

FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available to advance understanding of computer science, IT technology, economic, scientific, and social issues. We believe this constitutes a 'fair use' of any such copyrighted material as provided by section 107 of the US Copyright Law according to which such material can be distributed without profit exclusively for research and educational purposes.

This is a Spartan WHYFF (We Help You For Free) site written by people for whom English is not a native language. Grammar and spelling errors should be expected. The site contain some broken links as it develops like a living tree...

You can use PayPal to make a contribution, supporting development of this site and speed up access. In case softpanorama.org is down you can use the at softpanorama.info

Disclaimer:

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the author present and former employers, SDNP or any other organization the author may be associated with. We do not warrant the correctness of the information provided or its fitness for any purpose.

Last modified: August 06, 2017