Bash bug

This is a short (local) test of the bash bug, announced in September 2014. Check out the reference [vox.com] to learn more about the story behind the bug and its impact.

The machine runs Ubuntu Server 14.04 LTS.

$ uname -a
Linux amgspc 3.13.0-35-generic #62-Ubuntu SMP Fri Aug 15 01:58:42
UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

The distribution bash package is on version 4.3.11(1).

$ bash --version
GNU bash, version 4.3.11(1)-release (x86_64-pc-linux-gnu)
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

The bug, showing an arbitrary execution during an environment variable definition.

$ env AMGS='() { :;}; echo hey' bash -c "echo foobar"
hey
foobar

After update, the version number does not change. However, behavior does:

$ env AMGS='() { :;}; echo hey' bash -c "echo foobar"
bash: warning: AMGS: ignoring function definition attempt
bash: error importing function definition for `AMGS'
foobar

References

http://www.vox.com/2014/9/25/6843949/the-bash-bug-explained

Fonts and Matplotlib

There are a few posts on StackOverflow about changing the font used by matplotlib, but I was not able to find a practical solution, particularly if the font is not part of the current set of system fonts.

To use a new font:

  1. (Optional) Install the font in the system;
$ apt-get install msttcorefonts
  1. Copy the font to matplotlib font directory. In this case, it’s in a virtualenv (server);
$ cp usr/share/fonts/truetype/msttcorefonts/Times_New_Roman* ~/.virtualenvs/server/lib/python2.7/site-packages/matplotlib/mpl-data/fonts/ttf
  1. Clear the font cache;
$ rm ~/.cache/matplotlib/fontList.cache

The font should be available for plotting. Testing with the scatter example from matplotlib’s documentation:

import numpy as np
import matplotlib.pyplot as plt

plt.rc('font',family='Times New Roman')

N = 50
area = np.pi * (15 * np.random.rand(N))**2 
plt.scatter(np.random.rand(N), 
            np.random.rand(N), 
            s=area, 
            c=np.random.rand(N), 
            alpha=0.5)
plt.title("Sample title")
plt.ylabel("Random Value")
plt.xlabel("Random Variable")
plt.show()

Changing the font of a matplotlib plot

Note: If the font is not rendered, it may be because matplotlib needs to refresh its font cache (see Kevin’s comment):

import matplotlib
matplotlib.font_manager._rebuild()

Codesign gdb on Mac OS X Yosemite (10.10.2)

Mac OS X 10.10.2 does not come with gdb pre installed. It is available on homebrew:

$ brew tap homebrew/dupes
$ brew install gdb

The binary is installed on /usr/local/bin

When initializing gdb on a program (a.out) it will produce the following error:

    Starting program: a.out
    Unable to find Mach task port for process-id XXXXX: (os/kern) failure (0x5).
     (please check gdb is codesigned - see taskgated(8))
Start Keychain Access application (/Applications/Utilities/Keychain Access.app)

This error occurs because OSX implements a pid access policy which requires a digital signature for binaries to access other processes pids. To enable gdb access to other processes, we must first code sign the binary. This signature depends on a particular certificate, which the user must create and register with the system.

To create a code signing certificate, open the Keychain Access application. Choose menu Keychain Access -> Certificate Assistant -> Create a Certificate…

Choose a name for the certificate (e.g., gdb-cert), set Identity Type to Self Signed Root, set Certificate Type to Code Signing and select the Let me override defaults. Click several times on Continue until you get to the Specify a Location For The Certificate screen, then set Keychain to System.

Double click on the certificate, open Trust section, and set Code Signing to Always Trust. Exit Keychain Access application.

Restart the taskagted service, and sign the binary.

$ sudo killall taskgated
$ codesign -fs gdb-cert /usr/local/bin/gdb

ACM SIGSPATIAL 2014 @ Dallas, Texas

ACM SIGSPATIAL aka GIS 2014 conference notes

GIR 2014 Workshop (4/11/2014)

[Presented] Indirect Location Recommendation, André Sabino and Armanda Rodrigues. The paper describes the Productive Network model with a spatial extension.

Using Minimaps to Enable Toponym Resolution with an Effective 100% Rate of Recall, Hanan Samet

Spatial and Data Processing and Querying

TAREEG: A MapReduce-Based Web System for Extracting Spatial Data from OpenStreetMap, Louai Alarabi (University of Minnesota, USA), Ahmed Eldawy (University of Minnesota, USA), Rami Alghamdi (University of Minnesota, USA), Mohamed F. Mokbel (University of Minnesota, USA)

Recommendation and LBS

Parameter-Free Discovery and Recommendation of Areas-of-Interest, Dmitry Laptev (ETH Zurich, Switzerland), Alexey Tikhonov (Yandex, Russia), Pavel Serdyukov (Yandex, Russia), Gleb Gusev (Yandex, Russia)

Spatial Data Mining

Hourly Pedestrian Population Trends Estimation using Location Data from Smartphones Dealing with Temporal and Spatial Sparsity, Kentaro Nishi (The University of Tokyo, Japan), Kota Tsubouchi (Yahoo Japan Research, Japan), Masamichi Shimosaka (The University of Tokyo, Japan)

Dallas, Texas

The conference venue was the Renaissance Dallas Hotel, at the Dallas Market Center.

Hotel: Holiday Inn Dallas Market Center, 4500 Harry Hines Boulevard

Run: Sat, 8 Nov 2014 7:44 Central Time (US & Canada)

Distance: 6,56 km
Time: 40:09
Avg Pace: 6:07 min/km
Elevation Gain: 36 m

Dallas Run

Apache and PostgreSQL (postgis) for Django deploy in Ubuntu Server 14.04 (trusty) – Hidralerta Project

The Hidralerta project is deployed at http://giss02.di.fct.unl.pt/. This posts details the server setup and application deploy procedures.

Hidralerta is deployed on a Ubuntu 14.04 LTS server. We use Apache for both the application and its static files. The application runs on PostgreSQL database, with the PostGIS extension for spatial data support.

This is the current setup:

$ uname -a
Linux giss02 3.13.0-55-generic #92-Ubuntu SMP Sun Jun 14 18:32:20 UTC 2015 x86_64 
x86_64 x86_64 GNU/Linux
$ lsb_release -a
LSB Version:    core-2.0-amd64:core-2.0-noarch:core-3.0-amd64:core-3.0-noarch:
core-3.1-amd64:core-3.1-noarch:core-3.2-amd64:core-3.2-noarch:core-4.0-amd64:
core-4.0-noarch:core-4.1-amd64:core-4.1-noarch:security-4.0-amd64:security-4.0-noarch:
security-4.1-amd64:security-4.1-noarch
Distributor ID: Ubuntu
Description:    Ubuntu 14.04.2 LTS
Release:        14.04
Codename:       trusty

Apache setup

The server is not exclusive for this application. We’ll install LAMP to prepare for other projects that require MySQL.

$ sudo apt-get update
$ sudo tasksel

Install lamp server with tasksel.

tasksel_lamp

Install mod_wsgi

To enable Apache support for python code we’ll use mod_wsgi.

$ sudo apt-get install libapache2-mod-wsgi  

Now apache is able to load any python code with wsgi support, which in django applications is declared in the wsgi.pyfile,

Install Postgresql 9.3 and PostGIS 2.1 (http://www.postgresql.org/download/linux/ubuntu/)

Create the file /etc/apt/sources.list.d/pgdg.list, and add a line for the repository.

deb http://apt.postgresql.org/pub/repos/apt/ trusty-pgdg main

Import the repository signing key, and update the package lists.

$ wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add - 
$ sudo apt-get update 
$ sudo apt-get install postgresql-9.3 postgresql-9.3-postgis-2.1

Configure PostgreSQL

$ sudo vim /etc/postgresql/9.3/main/pg_hba.conf

#local   all   all   peer 
local   all   all   md5
$ sudo service postgresql restart

If you are having problems with the access, set the config to local all trust. It enables direct access from the local machine. Change it to md5 after the password is reset.

$ sudo passwd postgres
$ sudo su postgres 
$ psql
psql (9.3.2) Type "help" for help.
postgres=# \password
Enter new password:
Enter again:
postgres=# \q 

Create database and install the postgis extension.

$ psql -U postgres 
Password for user postgres: 
psql (9.3.2) Type "help" for help.
postgres=# CREATE DATABASE hidralerta;
CREATE DATABASE 
postgres=# \c hidralerta
You are now connected to database "hidralerta" as user "postgres".
hidralerta_django=# CREATE EXTENSION postgis;
CREATE EXTENSION
hidralerta_django=# \q

Setup pip and virtualenv

$ sudo apt-get install python-pip
$ sudo pip install -U pip
$ sudo pip install virtualenv virtualenvwrapper

Add the following to ~/.bashrc and reload the shell:

export WORKON_HOME=$HOME/.virtualenvs
. /usr/local/bin/virtualenvwrapper.sh

Reload the terminal, and create a virtualenv for the project

$ mkvirtualenv hidralerta

Create a working copy of the application

If git is not installed:

$ sudo apt-get install git

Clone the the project files from the bitbucket repository. You may need to get authorization from the project development team.

You need to register the www-data ssh keys with bitbucket. You’ll probably need to generate www-data user set of keys:

$ sudo -u www-data ssh-keygen -t rsa

The keys are stored at /var/www/.ssh.

To build the clone:

$ cd /var/www/
$ sudo -u www-data git clone [email protected]:andresabino/hidralerta.git

Commands executed in the project directory need the sudo -u www-data prefix.

Install project dependencies

Go to the project directory and install all requirements. These are maintained in the requirements.txt file.

$ pip install -r requirements.txt 

You may need to solve some external dependencies:

  • GDAL: must be installed before all requirements – $ sudo apt-get install gdal libgdal1-dev. The python package must be built with numpy support. Please install numpy before gdal. For details, please refer to the package documentation.

Build the database

To quickly build the database, use the fabric rule:

$ fab local_reset

This will create an admin user for you, and ask for a password.

Comply with Django’s deploy checklist

TODO

Setup Apache’s virtualhost for the project

Create a virtualhost for the server. This expect a local user amgs, which has the project virtualenv.

[/etc/apache2/sites-available/hidralerta.conf]

WSGIPythonPath /var/www/hidralerta/app:/var/www/hidralerta/app/lib:
/home/user/.virtualenvs/hidralerta/lib/python2.7/site-packages
WSGIScriptAlias / /var/www/hidralerta/app/hidralerta/wsgi.py

Alias /media/ /var/www/hidralerta/app/media/
Alias /static/ /var/www/hidralerta/app/static/

<Directory /var/www/hidralerta/app/static>
Require all granted
</Directory>

<Directory /var/www/hidralerta/app/media>
Require all granted
</Directory>

<Directory /var/www/hidralerta/app/hidralerta>
  <Files wsgi.py>
    Require all granted
  </Files>
</Directory>

Disable the default side, install the new configuration and reload.

$ sudo a2dissite 000-default 
$ sudo a2ensite www.example.com 
$ sudo services apache2 reload