Vanessa Binder

I have been visiting my sister last Christmas in Sheffield.

She does a wonderful art work and we have made a video to show it. We used my camera, Nikon D800E for filming.

I performed the  soundtrack.

You can have a look to her website: bagthecat.com

My brother in law has edited Fabrício

Check the video out

Some Recordings

Here is some tracks I recorder myself and put it on Soundcloud. I do not have much time to record things as I am not a musician for a living, but I have a very cool sound card from Focusrite, a tube pre-amp from ART and a mike, AKG c 1000s. I use REAPER as my recorder software on a MAC.

A free style performing a 10 strings Brazilian Folk Guitar, we call it “Viola Caipira” tuned in “Cebolão”.

A free style of kind of slide Blues

This is a free style Harmonica. I am using a Honner MarineBand.

This is a song I have recorded with a Gibson acoustic from 1965 and a Mandolin.

Intalling Django with Python33 on Eclipse with PyDev 64Bits

I had written a tutorial  to install django in eclipse with PyDev on windows 8. At that time, Django was advised to run over Python27. Another reason was Pillow library was not available for python3. Now, they are, so time to change it!

Well, Djanog 1.5 or later is now compatible with Python33, as you can check here: https://docs.djangoproject.com/en/dev/topics/python3/

I decided to use the 64bits version of python33 in my Windows 8.1, as my production Linux, where my application is running is a Linux 64bits. As you may know, the Linux world is much easier than windows flavors, that is why I am writing this post.

Its is pretty much the same as the last one.

First of all, you have to install Java JDK and Eclipse. The  eclipse edition I use is the Kepler Java EE developers 64bits.

Now you can install Python. It is a very good idea first unistall other python versions if you have some and delete the python folder by hand. You can donwload python33 64bit msi from python website: http://www.python.org/ftp/python/3.3.3/python-3.3.3.amd64.msi

In windows 8.1, the python msi only put the path for c:\python33 I had to put myself the one for the scrips. I delete all old python27 path I had.

PATH=PATH;c:\python33;c:\python33\Scripts

Installing things on python usually involves running python scripts. All we do is download the scripts and run them with the python interpreter. I like to save all these downloaded script straight in

c:\Python33

then run them from there, using the admin command prompt.

To install Django on windows, the best way is using pip, as they advise on Django website.

Before install pip, you have to install the setuptools. In these days, most of things you have to install in python use it.
Download the script from

https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py

and run it as

python ez_setup.py

To install pip, you have to download a python script from pip website.

https://raw.github.com/pypa/pip/master/contrib/get-pip.py

and then run

python get-pip.py

After that we should be able to install Django typing

pip install django

That is it. Django is installed. You should now be able to import django from python interpreter.

Now, open Eclipse, Help -> Eclipse Market place, search for PyDev and install it.

Then, go to Windos->preferences select PyDev and Python Interpreter

eclipse-pydev-setup

Press the button Auto Config. The options come checked already, but just make sure the folder site-packages are checked.
That is it. You can start coding django apps on Eclipse.

To use the admin facilities, you have to install the DB driver you are using. In my case, it is MariaDB, which should be MySQL like. I could not find any windows ready to go package.

I have found this:

https://github.com/davispuh/MySQL-for-Python-3

It does not install straight forward. After download the zip from GitHub, and decompressing, there is  file on the root folder, called:

setup_windows.py

You have to open this file and edit some lines.

There is a variable called mysql_root. I had to point this to my MariaDB folder.
The script could find any register location. there is file called “site.cfg”, it did not work even adding the key to the register. I just end up adding manually my MariaDB path in that line.

mysql_root='C:/Program Files/MariaDB 5.5'

And the include dir for the MySQL includes as well.
My MariaDB include folder has a diferent path.

include_dirs = [ os.path.join(mysql_root, r'include/mysql') ]

Your script is not ready just yet. You have to change one environment variable which C compiler to use. In my case, I use VS2012, but python looks for VS2010. All you need to do is install VS2010 Express or just run

SET VS100COMNTOOLS=%VS110COMNTOOLS%

It is a kind o “kludge”, but it worked for me.
After that, you go to the driver folder and run

python setup.py install

Some references:

https://docs.djangoproject.com/en/1.6/topics/install/#installing-official-release

http://www.pip-installer.org/en/latest/installing.html#using-the-installer

SQL Searching in All Tables

Today I had a good challenge with SQL query. My boss wanted a search facility in the extranet CRM we are working on. The way he wanted is like google. I mean, search in everything using some command options, like we do on google: “ ’cliente name’ + phonenumber
It needs to be parsed to an SQL statement using like similar to:

like  ‘%cliente name%’ and like ‘%phonenumber%’

I found a solution that is working with good performance. The platform here is MS SQL Server 2010 and C#.net 4.
We have many related tables with many fields to search in, starting from client names, telephone numbers, email addresses and so on, with many joins.
The best idea I had was put all searchable fields in a single table in a single field, like a mini data warehouse. That table would have the text field to search and the referent key to pick up the actual data after the search to send it to the user interface with a proper join SQL.
With this approach, a set of big queries including all searchable data and relationships tiding up and concatenating all the related data in a single text field
The mini data warehouse table is something like that:

create table dwt(
   recoveryky int,
   searchfield varchar(max)
)

You can put as many relevant keys you need depending on your data model. The concept of that, as I said above, is to be able to recovery the data with a normal join with that key later, to send it to the UI.
Then you can run as many selects and joins as you need to insert all the data in this table. The tricky bit here is null fields and numeric fields, as we have to use side joins as well and everything. We have to convert them and concatenate them all together.

They tell to avoid

SET CONCAT_NULL_YIELDS_NULL OFF

As you can read here

http://msdn.microsoft.com/en-us/library/ms176056.aspx

So, to make sure we got all data and everything on the place, I did the follow:

insert into dwt (rcoverykey,search)
select t1.key,
isnull(convert(varchar(max),t1.field1),'')
+ ' ' + isnull(convert(varchar(max), t1.field2),'')
+ ' ' + isnull(convert(varchar(max), t2.field3),'')
from table1 t1
left join table2 t2 on t1.foreignkey=t2.key
…..

I do not know if that is the most elegant solution, but it is working fine.
The database here is not that big (around 10000 records), so I can populate that table every time someone searches and use it as a temporary table, however you can run it separately to keep it update as a proper data warehouse.

Now we are ready to search. The search parameters are coming from the interface like that:

‘john barber 0789067’

With that, we have to write something like;

Select * from dwt where search like ‘%john%’ and search like ‘%barber%’ and search like  ‘%0789067%’

Changing the ands and ors depending on your criteria.
Two main things have to be done here, split the search sentence and dynamically build the SQL statement.
T-SQL does not have any array type and any built in split function. So, we have to build one Split function which returns a table. The table will be our array.
There are plenty of samples around.

CREATE function [dbo].[Split]
(
   @separator varchar(1),
   @param varchar(max)
)
returns @arrayTable table
(
    value varchar(max)
)
as
begin
    declare @i int

    while len(@param) > 0
    begin
        set @i = (case charindex(@separator,@param) when 0
            then len(@param)
            else charindex(@separator,@param) -1 end)

        insert into @arrayTable select substring(@param,1,@i)

        set @param = (case (len(@param) - @i) when 0
            then  ''
            else right(@param, len(@param) - @i - 1) end)
    end
return
end

and then we use this split function to build the dynamic sql

declare @sqlsearch varchar(max)

set @sqlsearch='select distinct key from dwt where 1=1'

declare @likeS varchar(max)

declare array cursor for select value from dbo.Split(' ',@keywords)

OPEN array
FETCH NEXT FROM array INTO @likeS

WHILE @@FETCH_STATUS = 0
BEGIN

set @sqlsearch=@sqlsearch + ' and search like ''%'+ @likeS +'%'''

FETCH NEXT FROM array INTO @likeS
END

close array
deallocate array

exec sp_sqlexec @sqlsearch

in the last part, then you can build the “likes” as you need depending on your search sentence.

In my case, as the front end is C#, I used a comma separated array. My C# code treats the sentence before calling the stored procedure.

Does someone have any better or elegant idea?

MySQL RENAME TABLE IF NOT EXISTS is not supported – Prestashop update

MySQL RENAME TABLE IF NOT EXISTS is not supported.

I had a problem today doing a massive update SQL script for Prestashop.

I could not do the update via 1-click update module. It gave quite a few of SQL errors.

So, I am getting the sql scripts and running them manually and fixing what is wrong. I do not know why, but my database seems to have some of the new tables
already. I suppose it was done for some modules or theme we have installed. I have no idea why!

Anyway, the point is I have to do the update. I am working in a test environment, of course, and the goal is to have a good sql script to run on the live one at the day of the update!

To do that, I wrote a little command prompt application in C# to read the script files from the install update folder. My application checks what my version is, in my case it is from 1.4.0.8 and I am going to 1.5.0.2, to pick up the right files.

Then, I have to do many adjustments on the SQL script like checking tables, replacing prefix and get everything ready to run smoothly.
The tricky one was the RENAME statements. Prestashop update renames some tables, but few of these tables are already in my data base.

MySQL does not support

RENAME TABLE IF NOT EXISTS

As you can see on: http://bugs.mysql.com/bug.php?id=4754

You can use the IF statement in MySQL in stored procedures. I was looking some ideas around and I found a creative guy using the PREPARE statement.
His solution is really cool and worked 100% for me.

SET @query = If((SELECT Count(*)
             FROM information_schema.tables
             WHERE table_schema = [DATABASE_NAME]
             AND table_name = 'video_top_day';)>0,
           'RENAME TABLE video_top_day TO video_top_day_for_delete',
           'SELECT \'nothing to rename\' status');

PREPARE stmt FROM @query;

EXECUTE stmt;

http://stackoverflow.com/questions/9279619/mysql-rename-table-if-exists

My C# code is replacing the RENAME statements with this, so I only rename the right tables

Thank you.

Install Django in Eclipse with PyDev on Windows 8

I have written another tutorial for Django 1.6 on Python33 on windows 8.1 64bit, check it out: http://www.rebojo.com/intalling-django-python33-eclipse-pydev-64bits/

 

I had to install django in eclipse development environment a few times this week, as I am updating my windows, laptop and office’s boxes to Windows 8. The last time I have done it was ages ago and I did not remember. So, I took some notes that can help someone else.

First of all, you have to install Java JDK and Eclipse. The eclipse edition I use is the Java EE developers.

Django runs with Python27. In windows 8, the python msi did not put any path for python.exe. I had to put myself. It is a good time to put the path for scripts as well. We are going to need it later for setuptools.

PATH=PATH;c:\python27;c:\python27\scripts

In windows 8, you access the System window right-clicking on the down-left corner.
Python and Eclipse installed and working, we are ready to go.
Installing things on python usually involves running python scripts. All we do is download the scripts and run them with the python interpreter. I like to save all these downloaded script straight in

c:\Python27

then run them from there, using the admin command prompt.

To install Django in windows, the best way is using pip, as they advise on Django website.

Before install pip, you have to install the setuptools. In these days, most of things you have to install in python use it.
Download the script from

http://peak.telecommunity.com/dist/ez_setup.py

and run it as

python ez_setup.py

To install pip, you have to download a python script from pip website.

https://raw.github.com/pypa/pip/master/contrib/get-pip.py

and then run

python get-pip.py

After that we should be able to install Django typing

pip install django

That is it. Django is installed. You should now be able to import django from python interpreter.

Now, open Eclipse, Help -> Eclipse Market place, search for PyDev and install it.

Then, go to Windos->preferences select PyDev and Python Interpreter

eclipse-pydev-setup

Press the button Auto Config. The options come checked already, but just make sure the folder site-packages are checked.
That is it. You can start coding django apps on Eclipse.

To use the admin facilities, you have to install the DB driver you are using. In my case, it is MySql.

You have to download the MySql-python and install it. For 32 bits you can do that through pip, but for 64 bits whiich is my case, I found this file that installs the thing easily.

http://www.codegood.com/download/11/

That is it. Just create a PyDev Django project, setup you DB details and you are ready to code!

 

References:

https://docs.djangoproject.com/en/dev/

WireShark on Debian as non-root user

Last week I needed a network sniffer to check what was going on with our IP packages. Something was wrong. The IP phones and asterisk was failing. For use VoIP and all its features, the network must to be 100%. As the problem happened randomicaly, the idea was to put an old laptop running wireshark on Debian as non-root user to capture the packages in the servers room. So we could see what was wrong and why.

I got the laptop, put debian squeeze on it with GUI and install wireshark using the command

apt-get install wireshark

When I launched the wireshark through the GUI and was about to setup the capture, I had a surprise! I could not see any NIC to setup the capture. Debian, by default, does not allow you to capture packages and use the NIC in promiscuous mode as a non-root user. What is a good thing. It is safe.

Wireshark uses /usr/bin/dumpcap program to do the capture. We have to give allow the user which runs wireshark to execute dumpcap with enough permissions to do the capture.

It is always a good idea to create a usergroup for those things, so if we have more users that need to do the same thing, we just add them to the group, and assign dumpcap to that group. We have to change its executable permissions to 750 too.

groupadd wireshark
chgrp wireshark /usr/bin/dumpcap
chmod 750 /usr/bin/dumpcap

I called my user “sniffer”.

usermod -a -G wireshark sniffer

Now, it is not ready yet. We have to tell debian that dumpcap can run called from a user within that group and its is able to do some tasks usually attributed to root users. One of those tasks are capture IP packages and do changes in the NIC mode.

That thing is called File Capabilities. We use setcap to do that. We have to install libcap2 and libcap2-bin if it is not already intalled using apt-get

apt-get install libcap2 libcap2-bin

And finally,

setcap cap_net_raw,cap_net_admin=eip /usr/bin/dumpcap

That was the difficult part. You can have a look in some Linux docs around the web to understand how and what capabilities are and how to use them. Those in my opinion are the safe ones.

Now we can run the wireshark as a non-root user!

References:

Wiki Wireshark

Chocolate Chip Muffins

This recipe for chocolate chip muffins can be used with berries or anything you want!

1+1/2 cup of flour
1/2 cups of sugar
3 tea spoons of baking powder
1/4 tea spoon of salt
1 cup of milk
1/3 cup of melted butter
1 egg
1 cup of chocolate chips

Mix the dry ingredients in a bowl.
Add the egg, the milk and the butter. Do not beat it. Just stir.

Put it in your muffin tray and then in the pre-heated oven.
The temperature of the oven should be about 190 degrees Celsius.
Bake it for about 20 minutes or until they go brown.

The recipe makes about 6 muffins.

Happy baking!

Linux useful commands

There are few linux shell commands that I do not use everyday, like the ones I need when the server crashes, and naturally, as the servers do not crashes everyday, I forget those commands and have to always google it again quickly. Another ones I forget are the ones I use for tuning. I do not tune servers everyday.

I put them here, so I do not need to find them out again.

Disk Usage:

df -h 

the -h give the information in Gb, otherwise, it would be displayed in bytes.

Number of CPUs:

cat /proc/cpuinfo

Mount a smb sharing from anothe Linux

mount -t smbfs  //<host>/<share> /mnt/<mountpoint>

If you have password in your share, then

mount -t smbfs -o username=<username>,password=<password> //<host>/<share> /mnt/<mountpoint>

Using tar to compress

tar -zcvf <destination_file>.tar.gz <source_folder>

Using tar to uncompress

tar -zxvf <file>.tar.gz

MySQL backup and restore

backup

mysqldump -u [username] -p [database] > filename.sql

restore

mysql -u [username] -p  [database] < filename.sql

Delete a non empty folder

rm -rfv [folder] 

Copying files and folders

cp -rv [source_folder]/* [destination_folder]/ 

Using Java Socket to dial out with Asterisk Manager Interface AMI

There are many ways to dial from Asterisk programmatically. It is really cool when your application through its own interface has the ability to dial out, ringing the user extension and the destination number.
One of these ways is using Asterisk Call files. All you have to do is copy a file in the format:

Channel: SIP/numbertodial
MaxRetries: 2
RetryTime: 60
WaitTime: 30
Context: outgoing-context
Extension: 100
Priority: 2

to

/var/spool/asterisk/outgoing

You can find documentation for that in:
https://wiki.asterisk.org/wiki/display/AST/Asterisk+Call+FIles

I was having performance problems with this method. I have made an Add-in for MS Outlook using this method and it was slow. It was taking long seconds to start ringing the user extension. It does not look nice!
To resolve that issue, I have made a Java application using sockets. It connects to Asterisk Manager Interface (AMI) to make the call. It is much quicker and it gives a very nice feedback from the users.
All you have to do is connect the AMI using a Socket and send the information to Asterisk using the AMI Syntax. You can find documentation in:

https://wiki.asterisk.org/wiki/display/AST/Asterisk+Manager+Interface+%28AMI%29

You have to enable and setup the Manager properly. See this link:

http://www.the-asterisk-book.com/unstable/asterisk-manager-api.html

import java.io.DataOutputStream;
import java.io.IOException;
import java.net.InetAddress;
import java.net.InetSocketAddress;
import java.net.Socket;
import java.net.SocketAddress;

public class Dialler {

	private static final String server = "localhost";
	private static final String user = "admin";
	private static final String password = "amp111";
	private static final int port = 5038;
	private static Socket socket;
    private static DataOutputStream wr;

	public static void main(String[] args) throws IOException {

		login();
		dial(args[0], args[1]);
               
	}
	private static void login() throws IOException {

		connect(server, port);
		send("Action: Login\r\n");
		send("username: " + user + "\r\n");
		send("secret: " + password + "\r\n");
		send("ActionID: 1\r\n");
		send("\r\n");
	}
	private static void dial(String extension, String number)
			throws IOException {

		send("Action: Originate\r\n");
		send("Channel: SIP/" + extension + "\r\n");
		send("Exten: " + number + "\r\n");
		send("Context: from-internal\r\n");
		send("Priority: 1\r\n");
		send("Async: yes\r\n\r\n");
	}
	private static void send(String request) 
                     throws IOException {

		wr.writeBytes(request);
	}

	private static void connect(String server, int port) 
                     throws IOException {

		InetAddress addr = InetAddress.getByName(server);
		SocketAddress sockaddr = new InetSocketAddress(addr, port);
		socket = new Socket();
		socket.connect(sockaddr);
        wr = new DataOutputStream(socket.getOutputStream());
	}
}

In my case, I do not need any feedback from asterisk, I just need to do a one go request and that is it!
If you need any feedback from AMI for exceptions treatment or a different functionality, you may use something like

DataInputStream wr = new DataInputStream(socket.getInputStream());
System.out.print(wr.Read()); //You should make a loop here.

If you close the streamers, you will close the socket. If you do that before asterisk to make the call, it will not work!
Using java socket to dial out with asterisk manager interface and adapting it, we can make a very sophisticated use of Asterisk in our application.
Please, let me know if you have any additional ideas or things that could work better!

« Older posts

Copyright © 2016 Juliano Binder