Αναζήτηση αυτού του ιστολογίου

Programming for Beginners - Python at Linux

Intro:
This is a small guide/tutorial for someone who needs to start programming in linux.
What ever will be presented bellow is actually personal research which i upload here for two reasons:
1. Not to be forgotten by myself.
2. It might help other newbie users to start programming in Linux.

Disclaimer: If you decide to use code from this post or from the links that this post will redirect, you have to do it at your own responsibility.

Basics:
Supposingly that we do know some basic programming from school and commands like if-then-else or variable declaration are understood, we will try to examine what is the situation today will all those suites, IDE, GUI, and so on.



Programming Language:
There are plenty of them, but in linux your system will have C, Python and Perl by default.

Python vs C:
Both work well with objects . We call objects in real life mainly for graphical applications (GUI).

C is the native language of Linux. The whole linux is built on C and it's varians (C#, C++, etc)
Making a C project you can just call existing C header and library files from your linux's heart.
But C is quite difficult for beginner.

On the other side, Python looks simpler than C, and reminds me the visual basic lessons in university (especially when combined with a nice IDE/GUI).
But Python can not directly import linux existing C libraries.

Python needs it's own libraries to talk to various system components, usually called bindings libraries and act like a bridge to Python and linux native libraries built with C.

For example:
evdev library provided by xserver packages in linux, talks with system devices using C.
To use this evdev library in your C code, you just import the library as it is.
But in Python you can not import xserver evdev C library.
For Python there is a python binding library called python-evdev.

So you have to install python evdev in your system, and then you can import this python evdev library in your Python project.

There are techniques to import (actually translate) C libraries in Python, and this is how you make a binding python library. But before go into those waters, search the web because it is more than 99% sure that python bindings for common linux libraries and basicall python operations (i.e device management, xserver, etc) already exist out there and you just need to install them.

Another difference is the speed. C is much faster than Python.
This is (probably) because C can talk with your system in lowere levels than Python.

Imagine as a rough example that you want to capture a key stroke or mouse movement.
With C you can capture this stroke on the time of generation by your hardware.
With python usually you catch this event in a higher level somewhere between the event chain (click -> kernel -> xserver -> screen display).

But since Python is easier i will start with Python and maybe i could switch later in C.


Interpretter and Compiler: 
The "engine" that will translate your code in something usefull for the pc.
You write human lines of code, but your pc undestands only 1 and 0.
These engines will translate your code.

Basic difference:
Interpreter , translates and runs the code line by line.
Compilers checks the whole code, detects errors (synthax errors, programming errors, etc) and then builts the code (if no errors), providing to you an executable file to run (i.e an exe file in windows).

Imagine a programm with 1000 lines and a synthax error in line 900. Interpretter will not understood that error unless line 900 is reached / called. Then will violently stop running your program.

Linux Python version 2.7 (Python 2.7 and 3.5 are installed by default in all distros), runs as #python myprog.py and uses an interpreter.

IDE's usually provide code verification even for interpreters to allow you to examine your code before running. Usefull when we use Python interpreter.

Also, there are tools like PyInstaller that are capable to wrap your programm and the corresponding modules / libraries that your program is based on, and generate an installation package with a simple file to run for the end user that will not have Python development libraries installed.

IDE: Is an active developers enviroment (PyCharm Edu, LiClipse, etc.)
IDE is a suite providing a text editor and other related stuff for your code.
You can write your code inside an IDE text editor and you can see the results of your code in a dedicated window inside your IDE, all in the same screen.

You could also write your code with plain text editors, save the file as myprog.py and then run the program in python (terminal window) using $python myprog.py - results appear in the terminal.

The difference with IDE is that provides all the necessary stuff in one place. Gives you one editor tab to write your code , gives you console window to see the resuls of your code, all in the same screen inside the IDE.

More over, using IDE you get code folding and support /suggestions/error checking for your code at the time of typing and not the time of code run (code verification before run, auto complete suggestions, properties browser, source browser. etc)

Imagine an object monaxoss, declared as human, with properties country, age, gender, stored in a library somewhere in your system.
By using a plain text editor you can import this library and object monaxoss, but you need to know exactly properties of this object in order to use it correctly in your code.
In a text editor you need to know that monaxoss.country will return you the value "Greece".

Using an IDE, when you type monaxoss. the time you press the dot a mini window appears presenting to you the available properties for this object to select (country, age, gender).

Another great feature i enjoyed in IDEs is that you can go to the source of the library you import with a single click in your code. I did a small project using Python and i imported a python library called pymouse.
By text selecting "import pymouse" in my code and then pressing F4 in my IDE, it took me to the source of pymouse (in a seperate tab,next to my code)!

Tips: How to install Eric6 IDE on Linux:
#apt install python3-pyqt5 python-qscintilla2 python3-pyqt5.qtsci python3-pyqt5.qtsql python3-pyqt5.qtsvg
Optional : #apt install python3-pip python3-setuptools
Once installed Eric6 is normally installed and be found on your apps-development. It is not running as a live / portable programm like PyCharm.


GUI : Graphical User Interface (i.e Glade)
Any window appears in your screen, any pop up window with warnings, any graphical dialogue , is a gui.Whatever is not "text" but it is graphic, it is a gui.

A GUI development suite could be generic (like Glade) or targeted to a specific language.
Can provide graphical design only or can combine gui and ide.
By the means of graphical design we mean that have a menu in your suite, you drag a window, you then drag a button inside the window, a dialogue and other relevan graphical elements.

If i'm not wrong, generic GUI Suites export the designed gui in xml format that can be bound with any language.

There are tons of GUI Suites.
Some of them use their own way to draw windows in your screen pixel by pixel, and some other use existed libraries in your system (i.e GTK libraries or Win32 Libraries) to draw a window, in the same way that your systemd does it (same format, color, theme, etc).

Framework : The word framework includes whatever provides a frame to work with tools/toolkits (ide. gui, etc)

Links:
LiClipse and PyCharm Edu , once unpacked run from the directory where unpacked without installation (like live / portable run).

Basic Dependencies to run Python:
  1. $ sudo apt-get install python-dev python-pip gcc
  2. $ sudo apt-get install linux-headers-$(uname -r)
Pypi and pip : https://pypi.python.org/pypi
Pypi is the Python Package Index and pip is a tool similar to apt / apt-get for installing packages from Pypi.
If your project needs evdev module, you install it using $sudo pip install evdev.
If your project needs pymouse : $sudo pip install pymouse.

In my system , i had to specify the --system option after install to force pip to install the python package in the system default directories for Python packages, otherwise Python was complaining about missing modules.

Tips:
Other pip command line arguments:
$sudo pip install,download,uninstall,search, list, show,
$sudo pip help + command, i.e pip help install : Gives you help about a command options
$sudo pip show package -> gives info about package , including package installation location
$pip install --upgrade package -> will upgrade a package (if new version is available).

User Rights - Access Write
Depending on the libraries used, it could required to run your script with sudo rights.
Linux is user focused system and it is advised to run Python as user and not using root account.
For special rights use sudo as a user. This help the whole story to store/retrieve data from the user locations.

For example , pip install package will install the python package in different location when the command run by the root account. Ths is why we have to use sudo pip , as user. Our user directories will be used.

Same story for IDEs. If you want to open your ide with special permissions do not use the root account but launch IDE software as sudo or gksu.


Scripts that require special actions like manipulating system devices (even to just read data) or manipulating other system components require special permissions.

For example the following script fails to run on PyCharm (as a user) giving an error related to dev variable (not defined or something like this):
dev = InputDevice(‘/dev/input/event7’)
capab = dev.capabilities(verbose=True)
print(‘Device Capabilities: ‘, capab)

But if you run this code as sudo, (either by user terminal as sudo python or by launching PyCharm with gksu ) the code runs without problems.

It is although strange why PyCharm did not warn me about access rights problem on the dev/input/event and complained about undefined dev variable.

In some other cases (i.e Geany Editor launched as user), i had to modify the built in execute command (available in options) from python %f to sudo python %f to be able to run the scripts as sudo with special permissions.

As a general idea when you open the ide or editor with sudo/gksu these special permissions are also applied to python command. But in any case we have to use gksu ide/editor instead of using the root account (i.e become super user with su command)

Python Eggs and Wheels:
http://stackoverflow.com/questions/2051192/what-is-a-python-egg
Python files are saved as .py and run as python file.py
A python egg file is a compressed (like zip) file including code and resources , in order to be redistributed.
Distributing an egg makes installation and update to target systems easy.
Egg has been replaced by "wheels".

Github , Git and Gist
https://help.github.com/articles/adding-a-file-to-a-repository-from-the-command-line/
http://stackoverflow.com/questions/1443210/updating-a-local-repository-with-changes-from-a-github-repository
Is quite common to use github and git in order to keep your code.
Putting your code in github , you can work from anywhere (i.e your home pc and your office pc) and all the code changes are stored in github. It works actually as a cloud , plus that you can allow other people to make changes to your code and commit these changes/patches.

Gist is a smaller area in github that you can insert some code snippets and get a link to paste the snippet in other pages (blogs,forums, etc).
Gist codes are not accesible by github but you can import your gists to github manually.
Check here: http://stackoverflow.com/questions/13671328/transfer-gist-repo-to-github

Git is the command line tool to manipulate your github files (apt-get install git)

Working with git:
1. Create a copy of the repository locally
#git clone https://github.com/gevasiliou/PythonTests.git
This makes a copy of the git repository in your local machine.
Although i run this command as root, i was glad to see that new folder PythonTests was created under /home/username folder instead of root folders.

2. Change file locally and comit changes to github
#git add . (mind the dot after add!)
#git comit -m 'Message like Updating Files"
#git push OR git push https://github.com/gevasiliou/PythonTests.git OR git push origin master

Tips: git add . and git comit , fully synhronizes local / remote repositories (all files). If you have manual add file in your local PythonTests folder, will be uploaded to github.
According to git help, it seems that with git comit --only you specify the file to comit.
With git comit --all you include everything, and with git comit --include you add a file to your comit (i suppose in the case you used --only before).

Moreover in git add, instead of dot which represents "all files and directories in this local folder", you can specify a particular directory /file to be added / prepared for commit.

I suppose that git comit --only has a meaning if you have make a git add . (all).
In my case, even that i specified git add myfile.py, the git comit upload everything that was present in my current folder (a couple of not relevant files). Or maybe i did something wrong, because the next time git add particular file worked ok.

3. Change file on github and update your local file
#git pull OR git pull https://github.com/gevasiliou/PythonTests.git OR git pull origin master

Remarks and tips:
To get rid of access rights and confusions to my python projects and git clones available in my local machine, i used to go in the settings of the new folder created (git clone) and change permissions to read and write access for all local users .

In case of #git push/pull origin master. master refers to your master branch of the last used repo.
After a first comit (from local to remote) it seems that your login details are kept and git uses the last commit settings to make a new commt.

Delete a repository: https://github.com/user/repositoryname/settings
It seems that there is not a link for repo settings and you have to manually enter the /settings in your browser. Scroll down to "danger" zone, and you will find a delete button.

Python Example 1 - Mouse pointer move and Click
Let's go deeper with Python (no, you will not see hello world examples here).
The first program is a Python 2.7 code, that will move your mouse pointer to the upper right corner of your screen and will perform a mouse button right click there.
from pymouse import PyMouse  #from pymouse library import PyMouse object / class.
#import sys
#print(sys.path)
# If you are sure that pymouse is installed but Python claims that is not found , print the system path to see what is considered to be
# default directories for Python extensions in your system and check if pymouse is located in one of these directories.
m = PyMouse()  # variable m is defined as a PyMouse object.
x,y = m.position() # x and y variables get mouse current position coordinates returned by m.position() in pixels.
print('Current position:',x,y)
a,b = m.screen_size()
print('screen_size:',a,b)  # The screen size is printed in pixels like 133 , 768
x=1360
y=5
m.move(x,y)  # move the mouse pointer to 1360,5
m.click(x,y,2) # third argument represents mouse button : 1 = left click, 2 = right click, 3=middle click

The comments should be enough to understand how this program works.
You need to install pymouse for this code to work:
$ sudo pip install --system pymouse python-xlib

Remarks:
1. pymouse is still available in pypi.
A newer version of pymouse together with python keyboard library is merged into a package called pyuserinput.  In any case, my code worked with the old pymouse package.

2. python-xlib (required by pymouse) is available also as a debian package and could be installed using apt-get. In any case, debian xlib is version 0.14, while pip xlib was version 0.17
Above programm will work no matter if you apt-get or pip install python-xlib .

Then Copy - Paste the above code in your text editor, save the file as test.py, and open terminal to run it : $python test.py

Program prints will be printed in terminal .
Watch the mouse moving and right - clicking at upper right corner of the screen.

Example 2 : Rigth Click in current mouse pointer position (simplified)
from pymouse import PyMouse
m = PyMouse()
x,y = m.position()
m.click(x,y,2)

Code 3: Simulating a right click on TouchScreens
There is a nice Python script that grabs the touchscreen, listens for touch events and raises a right click command in where the pointer is either with 1 finger long press or two fingers tap.

Original Script is made by a guy called Zyell , and works for Elan and Atmel Touchscreen.
https://github.com/Zyell/Python-Touchscreen-RightClick

Also there is a modification script here:
https://github.com/gevasiliou/Python-Touchscreen-RightClick

The original script uses python evded to give a right click command. The modified script uses pymouse library to send the right click even to the screen.

Moreover, the modified script applies a tolerance to the 1 finger long press of 50px, since in modern high resolution Elan Touchscreen the 1 finger touch, even if it looks steady (talking about coordinations) is not.

Above script of Zyell, gives a nice example of manipulating devices and grabbing generated events.

Code 4: Working with python evdev
Python-evdev Tutorial : http://python-evdev.readthedocs.io/en/latest/tutorial.html

4.1 : Listing Devices Capabilites usign python-evdev
from evdev import InputDevice, InputEvent, UInput, categorize, ecodes as e
#ecodes as e equals to e = ecodes
dev = InputDevice(‘/dev/input/event7’) # apply the event Number of the required device .
capab = dev.capabilities(verbose=True,absinfo=True)
print(‘Device Capabilities: ‘, capab)

4.2 : Listing all devices capabilities
from evdev import InputDevice, list_devices
from pprint import pprint #the pretty print modules
for dev in list_devices(): # scan all devices
   device = InputDevice(dev)
   pprint(('device: ', device,'-',device.name))
   cap = device.capabilities(verbose=True,absinfo=True)
   pprint(('Device Capabilities:', cap))

PS: pprint is usually installed by default with Python.

4.3 Inject an input event to the kernel using evdev/uinput
This makes use of the uinput, which is like a virtual input device - a user input.
You can create a uinput with default settings like ui=UInput() or you can set specific capabilities to uinput.
PS: there is also a seperate library called python-uinput , but in here i talk about evdev uinput method.

There is strange thing working with uinput. It seems that uinput can be created only for root account and not for sudo or gksu.

It seems that creating a uinput with out declaring it's capabilities is not enough to inject a mouse event. It should be , since luck of declaring capabilites adopts by default EV_KEY.

Also, it has been noticed that is not enough to declare basice capabilities like this:
from evdev import InputDevice, InputEvent, UInput, AbsInfo, categorize, ecodes as e
capabilities = {e.EV_ABS: (e.ABS_X, e.ABS_Y),
                e.EV_KEY: (e.BTN_LEFT, e.BTN_RIGHT)}
ui=UInput(capabilities) 

On the other hand, things work better when you declare detailed capabilities like this:
from evdev import InputDevice, InputEvent, UInput, AbsInfo, categorize, ecodes as e
capabilities = {
    e.EV_KEY : [e.BTN_LEFT, e.BTN_RIGHT],
    e.EV_ABS : [
         (e.ABS_X, AbsInfo(value=3100, min=0, max=3264, fuzz=0, flat=0, resolution=13)),
         (e.ABS_Y, AbsInfo(1090, 0, 1856, 0, 0, 13))]
}
ui=UInput(capabilities) 

In my tests, declaring detailed capabilites at abs_x and abs_y equal to my touchscreen properties (min-max-fuzz-flat-resoltion , listed by evtest or by code in 4.1 - 4.2 above)  made the following code to work:
ui.write(e.EV_ABS, e.ABS_X, 2000)
ui.write(e.EV_ABS, e.ABS_Y, 1000)
ui.write(e.EV_KEY, e.BTN_RIGHT, 1)  # KEY_A down
ui.write(e.EV_KEY, e.BTN_RIGHT, 0)  # KEY_A up
ui.syn()
#ui.close() #you could use ui.close but it seems not mandatory; ui is closed even with just ui.syn

I really saw the mouse pointer to be moved in the touchscreen and i indeed saw a right click!

With above tips, i was able to simulate a right click in a virtual box machine , with mouse integration (target machine is configured as vbox tablet) using this code:

from evdev import InputDevice, InputEvent, UInput, AbsInfo, categorize, ecodes as e
import evdev
import time
device = InputDevice('/dev/input/event2') # adjust the correct event number
print(device)
cap = device.capabilities(verbose=True,absinfo=True)
print('Device Capabilities:', cap) # Prints device capabilities in format Type : Values
capabilities = {
    e.EV_KEY : [e.BTN_LEFT, e.BTN_RIGHT],
    e.EV_ABS : [
         (e.ABS_X, AbsInfo(value=5000, min=0, max=32767, fuzz=0, flat=0, resolution=0)),
         (e.ABS_Y, AbsInfo(5000, 0, 32767, 0, 0, 0))]
}
#Capabilites applied match capabilities of vbox table device , as listed by evtest.
ui = UInput(capabilities)
print('UInput Capabilites:', ui.capabilities(verbose=True,absinfo=True))
time.sleep(15)
ui.write(e.EV_ABS, e.ABS_X, 25500)  # random coordinates for testing
ui.write(e.EV_ABS, e.ABS_Y, 14500)  # random coordinates for testing
ui.write(e.EV_KEY, e.BTN_RIGHT, 1)
ui.write(e.EV_KEY, e.BTN_RIGHT, 0)
ui.syn()
ui.close()
print('event injected')

Alternativelly, even this method works for injecting a right click event.
By writing 0 at abs-x and abs-y , right click in injected to wherever position the mouse point currently is.

ui.write(e.EV_ABS, e.ABS_X, 0) #this also works: Right click injected at wherever mouse it is
ui.write(e.EV_ABS, e.ABS_Y, 0) #this also works: Right click injected at wherever mouse it is.
ui.write(e.EV_KEY, e.BTN_RIGHT, 1) 
ui.write(e.EV_KEY, e.BTN_RIGHT, 0) 
ui.syn() 
ui.close() 

Python evdev/uinput Troubleshooting:
One way to troubleshoot uinput is to use two different root terminals (become root with su), assuming that your code is ok , python is not complaining but you don't see the expected results.

Moreover you need to apply a time delay from uinput creation and uinput.write events.
As a rough guide you do:
Terminal 1 : you run your file #python myfile.py
Once userinput is created , sleep timer starts counting
Terminal 2 : You run #evtest
You should use a virtual device in the end of the list with a name python-evdev...
Select this device at the evtest prompt .
Wait for the sleep timer to finish, and event should be injected by your python script.
If the event is injected , values abs_x, abs_y , btn_right, will change in terminal 2, which had remained in listening state.
Since the uinput is closed, evtest will complain about missing device.

5. Error Handling : 
You do error handling with try and except.
The except section has a logical meaning closing to EXPECT.

In the modified code of TouchScreenRightClick , the programmer wrote:
try:
    self.dev.ungrab()
except (OSError,IOError):  # capture case where grab was never initiated
    pass

This code just make a pass = go on , if OSError or IOError appears under the "try".
PS: It is true that a device ungrab in an already released device raises an IOError.

6. If _name_==_main__ explanation
The best explanation for this statement which exists almost in every Python program, is this example taken by http://stackoverflow.com/questions/419163/what-does-if-name-main-do

Actually the _name_ is a variable with a value.
Value can be   _main_ when the script runs directly by it self .
If the script is called by another script the _name_ will get as value the name of the script. 

Create the following two seperate files.
# File a.py
import b
#End of File a.py
AND also create
# File b.py
print "Hello World from %s!" % __name__
if __name__ == '__main__':
    print "Hello World again from %s!" % __name__
Running them will get you this output:
#End of File b.py

Now Run:
$ python a.py
Output -> Hello World from b!
As you can see, when a module is imported, Python sets globals()['__name__'] in this module to the module's name. As a result, the second half of the file b.py (Hello world again) is not executed since file b.py didn't run directly but was called as a module by file a.py

If you now run directly the file b.py:
$ python b.py
Output -> Hello World from __main__!
Output -> Hello World again from __main__!

As you can see, when b file is executed, Python sets globals()['__name__'] in this file to "__main__".

Obviously by controlling / checking the value of the variable _name_ you can execute different code if your script is inserted in module or it is run directly.


7. Teminating - Interupting a program / function:
A lot of options available according to this and even this post.
This commands terminate the intepreter : import sys and sys.exit , quit(), exit() , raise SystemExit
PS: If you terminate the interpreter and your script has been called by another script, BOTH scripts will stop.
Alternative method:
 def some_function():

     if do_not_continue:
         return    # implicitly, this is the same as saying return None

     do_something_else()
Or can be combined in the opposite way :
if not do_not_continue:
       
else:
       return

8. Defining a simple boolean Function
def check_winner(): 
    check something
    #here needs to be an if / then statement deciding if the game is over, return True if over, false if not
    if score == 100:
        return True
    else:
        return False

*. Other Cool Stuff
*.1 Get user id : os.getuid -> gives info about user id (0=root)

*.2 Define user rights using error handling (!)
try:
    os.rename('/etc/foo', '/etc/bar')
except IOError as e:
    if (e[0] == errno.EPERM):
       print >> sys.stderr, "You need root permissions to do this!"
       sys.exit(1)