Reducing the Memory of Apache

I have been running a bot on a Linux cloud service.

A quarter of the memory was from the apache2 process.

 

Reduced it by changing maxclients to 1, as only I use it.

That was done by adding these lines to my /etc/apache2/apache2.conf file:

# prefork MPM
# StartServers: number of server processes to start
# MinSpareServers: minimum number of server processes which are kept spare
# MaxSpareServers: maximum number of server processes which are kept spare
# MaxClients: maximum number of server processes allowed to start
# MaxRequestsPerChild: maximum number of requests a server process serves
<IfModule mpm_prefork_module>
StartServers 5
MinSpareServers 5
MaxSpareServers 10
MaxClients 1
MaxRequestsPerChild 0
</IfModule>

And then running

    apache2ctl graceful-stop && apache2ctl start

Source: http://www.inetservicescloud.com/knowledgebase/setting-maxclients-in-apacheprefork-mpm/

 

Searching / Finding Files Containing Text

grep -rnw helloWorld | grep src/ | cut -c -200

What to do if logging in to Ubuntu Unity Desktop Fails

I occasionally have an error where:

– I try to log into Ubuntu Unity Desktop
– I type in password successfully
– After a few seconds the screen flickers and I am taken back to the login prompt.

This problem is caused by an NVidia driver problems. For some reason it can go missing.

Fix by:

– Going to TTY
– Logging in

sudo service lightdm stop

– Go to my installation files folder, and running the NVidia driver installation file
– When it asks if I want to make changes to XConfig, say no, or it will break the system and you’ll get lots of boot problems.

Using SQL Alchemy With New Server Bot

Had a problem using SQLAlchemy in Python 3.5, as MySqlDB is not in Python 3.x.

The solution is to specify pymysql in the connection string. E.g.

mysql+pymysql://user:password@localhost/database

To set the user, I went to the users and priveleges page of mysqlworkbench. Will I be able to use Mysql workbench for my Linode server?

Making Cuda Work With Numba

Add these lines to .bashrc:

export NUMBAPRO_LIBDEVICE=/usr/local/cuda/nvvm/libdevice
export NUMBAPRO_NVVM=/usr/local/cuda/nvvm/lib64/libnvvm.so.3

More info:

http://stackoverflow.com/questions/41496299/library-nvvm-not-found-when-trying-to-use-numba

Example Crontab

An example crontab for my Linode box, which runs scheduled Python scripts.

The PATH and PYTHONPATH lines set environment variables. The scripts reside in /srv/, rather than /home, as I also use apache, which would have trouble reading scripts from /home.

PATH=/opt/anaconda3/bin
PYTHONPATH=/srv/Project:/srv/StateMachine

*/15 * * * * python /srv/Project/bot/scripts/keep_alive.py
*/15 * * * * python /srv/Project/bot/scripts/download_data.py

Ubuntu Things To Do After Install

A list of things to do when installing my dev machine, which took me a week and a half to work out due to NVIDIA drivers causing boot issues. So I want to document it here.

The purpose of the box is quantitative Python development, hopefully using some AI.

Software To Install

1. Install NVidia Drivers:
   No need to purge the drivers as nothing will be installed yet.
   Check that it boots afterwards.
   https://quantitativenotes.wordpress.com/2017/01/01/installing-nvidia-drivers-on-ubuntu/
2. Install Cuda:
   No need to purge the old drivers as we will only have the one that we want. Don’t let it install the driver!

   Fixing GPU Cuda

   
   http://askubuntu.com/questions/799184/how-can-i-install-cuda-on-ubuntu-16-04

3. Install DropBox:
   https://www.dropbox.com/en_GB/install-linux
4. Install Anacondas:
   https://www.continuum.io/downloads
   Install keras, which also installs Theano and Tensorflow:
   conda install keras
   Add these lines to .bashrc:
   export NUMBAPRO_LIBDEVICE=/usr/local/cuda/nvvm/libdevice
   export NUMBAPRO_NVVM=/usr/local/cuda/nvvm/lib64/libnvvm.so.3
   export PATH=/usr/local/cuda/bin:/usr/local/cuda/lib64:$PATH
   export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
   
5. Install PyCharm
6. Install Virtual Box
7. Install git and enable credential caching:
   git config –global credential.helper ‘cache –timeout 3600’

Ubuntu Customizations that I prefer:

1. Add a keyboard shortcut to Nautilus:
   http://askubuntu.com/questions/483917/is-there-any-shortcut-keys-to-open-nautilus
2. Display lists, rather than icons, in Nautilus:
   http://askubuntu.com/questions/134371/how-do-i-set-default-view-to-list-in-nautilus-file-manager
3. In Appearance/Behaviour, “Show the menus for a window” “In the window’s title bar”, or you might get confused when they appear in the top left of the screen.

Installing NVidia Drivers on Ubuntu

https://elementaryforums.com/index.php?threads/howto-install-latest-nvidia-driver-on-linux-without-getting-black-screen.7/

The important bit:

#Would you like to run the nvidia-xconfig utility to automatically update your X Configuration file so set the NVIDIA X driver will be used when you restart X?

#no,no,No,NO,NO!!!

Fixing GPU Cuda

I had this error:

ERROR (theano.sandbox.cuda): Failed to compile cuda_ndarray.cu: libcublas.so.6.5: cannot open shared object file: No such file or directory WARNING (theano.sandbox.cuda): CUDA is installed, but device gpu is not available (error: cuda unavilable) [Elemwise{exp,no_inplace}(<TensorType(float32, vector)>)]

When running:

THEANO_FLAGS=mode=FAST_RUN,device=gpu,floatX=float32 python check1.py

I fixed it using a tip from here, adding the following lines to bashrc:

export PATH=/usr/local/cuda/bin:/usr/local/cuda/lib64:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH

 

I installed Cuda using these instructions:

http://askubuntu.com/questions/799184/how-can-i-install-cuda-on-ubuntu-16-04

Did accelerated drivers break my build?

When installing cuda, it asked if I wanted to install accelerated drivers. I said yes. I think that might have broken my build as it worked when I installed the stuff manually.

However…

***WARNING: Incomplete installation! This installation did not install the CUDA Driver. A driver of version at least 361.00 is required for CUDA 8.0 functionality to work.
To install the driver using this installer, run the following command, replacing with the name of this run file:
sudo .run -silent -driver

It was NVIDIA-Linux-x86_64-375.26 that fixed my machine. Why is it broken?