Using Syntastic for Python development

I use Synstastic plugin of vim for syntax checking in vim. Syntastic offers syntax checking for a LOT of languages. But, there is a problem that i had been facing with it. For a file with larger than 4k lines, it takes a lot of time to check the syntax and it used to happen every time you save the file. Syntax checking on write operation is the default behavior.

So, i did some changes in my .vimrc so that i could still use Syntastic for larger files. Do note that syntastic checking still takes a long time but, i have configured it to be called whenever i want to rather than on every write operation or opening of file.

” show list of errors and warnings on the current file
nmap <leader>e :Errors<CR>
” Whether to perform syntastic checking on opening of file
” This made it very slow on open, so don’t
let g:syntastic_check_on_open = 0
” Don’t check every time i save the file
” I will call you when i need you
let g:syntastic_check_on_wq = 0
” By default, keep syntastic in passive mode
let g:syntastic_mode_map = { ‘mode’: ‘passive’ }
” Use :Sc to perform syntastic check
:command Sc :SyntasticCheck
” Check pylint for python
let g:syntastic_python_checkers = [‘pylint’]
” For jsx – React and React native
let g:syntastic_javascript_checkers = [‘eslint’]

This change made opening of a larger python file ~25s (yes, seconds) faster. It still takes a lot of time for syntax checking though. I will have to find out why and if i could do anything about it. I don’t want to leave out this plugin because it offers so much. I could simply use Python-mode for python syntax checking but, what about the rest of the languages which i am going to use.

Sending Emails using Django and Sendgrid

Recently, i setup a django app which uses  sendgrid to send emails. I will go through the steps in this short blog.

  1. Register at sendgrid
  2. Choose SMTP for sending emails
  3. Get an API key. The last step will redirect you to this. This key will also be your password. The username that they gave me was apikey so, i guess this remains same for everyone.
  4. Configure your django settings to this:
    EMAIL_HOST_USER = ‘<your username here>’
    EMAIL_HOST = ‘smtp.sendgrid.net’
    EMAIL_HOST_PASSWORD = ‘<your password here>’
    EMAIL_PORT = 587
    EMAIL_USE_TLS = True
    EMAIL_BACKEND = ‘django.core.mail.backends.smtp.EmailBackend’  (this is the default value of EMAIL_BACKEND btw)
  5. Use django.core.mail.send_mail for sending emails now

Django with uwsgi and nginx on fedora

Today, i deployed a django project using uwsgi and nginx on a fedora 26 instance on AWS. I will talk about the same here.

I had used gunicorn in the past but never uwsgi. Getting started with gunicorn was a little bit easier for me than uwsgi primarily because i didn’t know i had to install uwsgi-plugin-python package to use uwsgi with django. This took me a while because, there were no errors. There was a “no app could be loaded” problem but, on internet most of this kind of error is for flask. Flask exposes it’s application object as app and uwsgi looks to load application which it fails to find.

The steps are:

  1. Install dependencies and the project itself:
    • sudo dnf install uwsgi uwsgi-plugin-python nginx
  1. Create a configuration file for uwsgi: uwsgi.ini
  2. Change nginx config to pass on to uwsgi for incoming requests on /ethcld/ (the mount point that i used)

 

Here is my uwsgi file:

[uwsgi]
chdir = /home/fedora/ethcld
plugin = python
# Django’s wsgi file
# module = ethereal.wsgi:application
mount = /ethcld=ethereal.wsgi:application
manage-script-name = True
master = True
# maximum number of worker processes
processes = 4
# Threads per process
threads = 2
socket = 127.0.0.1:8001
# clear environment on exit
vacuum = true

uwsgi asks for the directory of the project. In my case, it was /home/fedora/ethcld/. Mount is optional, if you want to run the application under some namespace, you will have to use mount. Also, if mount is getting used, you should not need module.

Manage Script name (manage-script-name)is important while mounting otherwise you will get a bunch of 404s. Usually, the request comes for something like: GET /username/  but, without manage-script-name option, nginx will not map /username/ to /ethcld/username.

For socket, i could have used unix socket instead of port one. But, somehow i settled for port.

On the nginx side, i did the following changes:

http {

       include /etc/nginx/mime.types;

       server {

                location ^~/static/ {
                       autoindex on;
                       alias /opt/ethcld/static/;
                }

                location ^~ /ethcld {
                        include uwsgi_params;
                        uwsgi_pass 127.0.0.1:8001;
                }

         }

}

Nginx supports uwsgi protocol by default, so options are already there, we just need to call them. For serving static files, for django, it is recommended that we do:

  • python manage.py collectstatic

This will copy (by default) all the static files in the django app to the location specified in STATIC_ROOT in settings. From there we can serve the static files. There are possible optimizations that can be done like gzip. But, i did this for testing only.

You need to include mime.types, otherwise browsers will keep rejecting files. By default, the mime type is ‘text/plain’.

Using IRC from Mobile

In this blog post, i will talk about how i use IRC from mobile.

I have been using weechat as my irc client for about a year now and i am happy with it. Generally, i used to open weechat in one tmux session and project (or projects) in other tmux sessions in my machine and stay on IRC while working. This serves well except for the fact that when you are not connected with internet, nobody on irc can leave a message for you.

I didn’t know about weechat’s relay feature until very recently when maxking was talking about this in #dgplug. In this feature, weechat listens to client connections to a specified port that allows two way communication between connected clients and weechat. The clients can be any device with internet access. The only thing left then for this setup to work was an android app. There were two options, glowing bear and weechat-android. The former didn’t support ssl so it was out of picture.

A few weeks ago, i got a free tier fedora 26 instance on AWS. I had to use this for testing other applications that i was working on. Also, AWS doesn’t allow(atleast for my instance) HTTPS connections for ports other than 443. I wanted to use ssl and that too for more than one application and thus decided using nginx as reverse proxy.

Here are the list of things that i did:

  1. Installed nginx, tmux and weechat on aws instance.
  2. Created self-signed certificate and pointed nginx to that.
  3. Configure weechat to relay on port 9001 and configure nginx for websocket connections on 443 and proxy it to 9001.
  4. Use weechat-android to connect to the relay using websocket(ssl) as connection type.

I used tmux so that i could ssh into the aws instance and join the tmux session. This is where the client-server architecture of tmux helped. I couldn’t use let’s encrypt or ACM for ssl certificate because i didn’t have domain name for that public IP. Creating self-signed certificate is surprisingly easy and this Digital Ocean blog helped.

I also use urlserver.py plugin for weechat which shortens url for you. It runs a small server on the system and provides redirects to the original link. With nginx, i am able to configure urlserver.py to run on one port and point nginx to that.

I am not fully happy with weechat-android though. Most of the time it works but, sometimes it disconnects right after connecting without saying anything. Other people have found this too here. Atleast, people can drop any message for me now.

PyCon 2016

I come from a place  where everyone worships competitive coding and thus cpp, so the experience of attending my first pycon was much awaited for me.

This year’s PyCon India happened in Delhi and i along with a couple of my friends reached on 23rd September, the first day. We were a bit late but it was all right because, we didn’t miss anything.

Day 1

We had workshops and devsprints on the first day. I, along with Farhaan, were taking a devsprint for Pagure project. It was nice to see a couple of new contributors whom i meet on IRC asking for help and trying to make a contribution. It went all day long but i did manage to roam around a little.

Sayan came to the spot where we were sitting for devsprint with a camera in hand. I don’t know if he was hired as an official photographer for PyCon or not but if he wasn’t i am sure he must have clicked more photographs than all the rest of people combined.

I am not a sort of person who meets a lot of people. I, generally, feel awkward meeting new people and it’s not easy for me to get comfortable with anyone. With Farhaan and especially Sayan, this wasn’t the case. They made me comfortable in our first encounter. In fact, the most shocking thing about PyCon was simplicity of the people. I expected them to be nice but they were better. 🙂

I then attended Pycharm workshop. Sayan was sitting in the first row and i and Farhaan joined him there. This workshop turned out to be really funny because we were comparing how we did things in Vim to the corresponding method in Pycharm. Pycharm does make things a little easier but moving the hand to mouse/touchpad for every little thing is too much to ask. We came to know that Sayan uses arrow keys instead of h/j/k/l in Vim (and he told me not to judge him for this :-p).

At the end of day one, I, my friend Saptak(saptaks), Farhaan, Sayan and Subhendu (subho or Ghost-Script) decided to visit Ambience Mall which was nearby, for some food. We ended up at KFC where we spend some time and  got to know each other better. Sayan told us about his academic history and  Hackerearth. We also talked about his experience of flock this year. After this, Sayan told me that i was a completely different person than i thought i would be :-p .

Day 2

It was a busy day for everyone. There were a lot of interesting talks. I managed to get a ticket to PyCon for one of my friend who sat  all day, the day before in the hotel room. At the end of the last day, i had joined the volunteers group since, most of the dgplug members were in the volunteers group and i didn’t want to miss anything. So, i was basically moving around during the talks doing small things. I couldn’t really concentrate on any particular talk except talk whose title read: ‘Helix and Salt: Case study in high volume and distributed python applications’, given by a linkedin guy named Akhil Malik and i didn’t understand much. At the end of the talk, Saptak asked me if i wanted to have a cold drink to which i replied: “I will need a harder drink than that.”. I didn’t realize Kushal was sitting just a row behind me and could easily have listened to this conversation.

At the end of second day, all the volunteers were supposed to have dinner together and i was supposed to meet a close friend of mine who lives in Delhi. Thankfully, I managed to do both but, i missed the starters :/ . The four of us from last night were joined by Farhaan and since, me and Farhaan both were present, the talk naturally shifted to our GSoC experiences. Sayan told me about writing blogs more often and he did have some valid points.

cttei5euaaawhj4-jpglarge

Day 3

 It was the last day and somehow i was feeling a little low because… i didn’t want it to end. I wasn’t interested in talks anymore. But, i did attend the lightening talks and was roaming rest of the time. There was a dgplug “staircase” meeting which i attended. Kushal was leading the talk and he was surrounded by about 30 people, most of whom hadn’t started their FOSS journey. He talked mainly about how they should start with something small and they will get better. It is a really nice initiative, i personally feel.

I had met Kushal last night at dinner and he said he had something to give to me and Farhaan. Just before the lightening talk, i was sitting in the second row and he came to me and stood by my side. He told me to be seated and gave me dgplug sticker and a set of stickers of fedora. This was a nice moment for me. Earlier that day, he had mentioned me and Farhaan for our contributions to pagure in his talk. In the small period of time that i have seen or been with Kushal, he has managed to earn a lot of respect from me.

At the end of this day, everybody had to leave so, the four people from first day and one other friend of mine, decided to visit the same mall once again, for food and roaming around. The Cab driver that we booked turned out to be very patriotic and got offended by my comment that we didn’t study at JNU (where the event was held) and we were “foreigners”. He kept talking about me being a foreigner the whole ride, no matter how many times i said i didn’t mean it. Obviously, everyone enjoyed it.

At the mall, there were no juniors or seniors. There were just 5 (single) computer science guys: being mean, pulling each other’s legs and talking about stuff i shouldn’t mention here. It turned out that Sayan and Subhendu got to know a few of my negative points as well. Sayan also managed to ask Saptak, me and Shubham (my other friend) to start contributing to fedora hubs.

Overall, it was a great experience meeting people whom i talk to on IRC. I won’t be able to mention all the people whom i have met during the event but it doesn’t matter. The important thing is that i enjoyed a lot and i am able to connect with them better.

GSoC Wrap Up

GSoC 2016 finished last week and i am writing this blog to list the work done by me in last three months for Fedora. My project was to adjust pagure and write script(s) so that we can have pkgs.fedoraproject.org on a pagure instance. We have it in staging currently http://pkgs.stg.fedoraproject.org/pagure/

https://pagure.io/pagure/pull-request/1007

https://pagure.io/pagure/pull-request/1035

https://pagure.io/pagure/pull-request/1036

https://pagure.io/pagure/pull-request/1045

https://pagure.io/pagure/pull-request/1050

https://pagure.io/pagure/pull-request/1058

https://pagure.io/pagure/pull-request/1071

https://pagure.io/pagure/pull-request/1094

https://pagure.io/pagure/pull-request/1095

https://pagure.io/pagure/pull-request/1097

https://pagure.io/pagure/pull-request/1114

https://pagure.io/pagure/pull-request/1120

https://pagure.io/pagure/pull-request/1149

https://pagure.io/pagure/pull-request/1150

https://pagure.io/pagure/pull-request/1151

https://pagure.io/pagure/pull-request/1157

https://pagure.io/pagure/pull-request/1177

https://pagure.io/pagure/pull-request/1210

https://pagure.io/pagure/pull-request/1211

https://pagure.io/pagure/pull-request/1219

https://pagure.io/pagure/pull-request/1218

https://pagure.io/pagure/pull-request/1158

Besides these, there is a script for getting user acls from pkgdb:

https://infrastructure.fedoraproject.org/cgit/ansible.git/commit/?id=de67bcbea22bb4539e32d195a10448948bc6d765

For me, the experience has been perfect. I like the work environment at #fedora-apps. My mentor, Pierre-Yves Chibon is nice to everyone and i hope i haven’t annoyed or disappointed him in last 3 months. It’s hard to find a person who can guide so patiently. I am saying this not because i see one of my friend working for FOSS Asia but, because he is genuinely good.

If GSoC wasn’t there, even then i would have spent my last 3 months in the same way (without mine and my father’s new mobile phone). I contribute here because i like the work environment that they have created and i get to learn new things while working on real life projects.

So, thanks Google for the money and Fedora for such an awesome experience.

User’s project watch list on pagure

Not long ago, Gaurav added watch feature on pagure. But, It had one thing missing from it: a user could not see what all projects he/she is watching. So, with pull request #1158, i tried to solve that problem.

For those of you are not aware of this feature, a user can now subscribe for emails for development of a project. He/She will receive emails for any changes on the issue tracker or if anything happens in any pull request. By default, the admin of a project is watching the project. However, if he wishes, he can unwatch it and he won’t receive emails for that project anymore. This can help in situations when the user is admin of a lot of projects and is no longer interested in some of them.

I won’t go in details of how this was implemented since, it involves simple function calls and minor addition in the UI (plus, there is a link to the PR). I will, however, attach screenshot of how it will look, when it will be live on pagure.io.

Screenshot from 2016-08-16 17-03-29