Things Do change with Time

Law of nature is to change, so true it is. I always worry about concepts which I didn’t grasp easily and thing that bother me a lot is how will I understand them, learn them, if no one is there to help as all people around also don’t know much about it; Finally reached to conclusion that if we give two, one or even half hour daily to that particular topic even if we don’t understand much about it soon we will become expert in it. Your enemy will become your friend if you keep try for same.

New things are same for all, no one come in this world with full stack of knowledge, every person learn every thing on this earth only. The point is that the more time you will spend in a thing the more you will know about it, as you will have experience you get knowledge in it and become expert. So the topic/concept which was new and tough to you will be no more if you spend time on it.

Don’t take my words take your own example: you know very well how to add or multiply two numbers(correct?), how you learned them? May your teacher helped you initially to know the technique of performing addition/multiplication but ultimately you practice to learn it. And does that teacher is still with you to teach each and every new thing or do you have guaranty that, that teacher will remain with you forever? Absolutely not. You should be capable enough to find your own ways to get the things done.

In order to learn new concept/topic very first thing I do is use internet wisely, gather upper level knowledge of that topic through internet by reading various posts but as we can’t trust each and every information present of internet, one need to put more efforts to become expert and also you can’t be expert in any thing if you don’t do it yourself.  So after gathering top level knowledge on that topic I search for any online course/MOOC and read good books to get well sequenced and trusted knowledge.

So the point is that things doesn’t change themselves with time, they change by doing things. Now doer could be you or other who will change it!

Things Do change with Time

Playing with QueueController

My current task is to shift QueueController from Data to Control plane, at present it is in data plane[1] But I am stuck in between and here putting the situation. flaper87, vkmc and I are thinking to get the solution. Here I am putting my thoughts and efforts for same. Btw you already know flaper87 is my mentor and vkmc is another OpenStack hacker.

So I read code of different classes that reside under zaqar/storage like DriverBase, DataDriverBase, ControlDriverBase, Queue etc; both control and data plane are part of storage layer of zaqar, so we need to look there only atm. For this task main class at present is DataDriverBase and we need to shift its queue_controller()method into ControlDriver. This queue_controller which is inside, is an abstract method. So I went to look for its implementation in the classes that have inherited DataDriverBase. And found that of all different datastores which zaqar support atm, that is mongodb, redis and sqlalchemy, have implementation of queue_controller inside driver.DataDriver class and also and have its implementation which are inside same package zaqar/storage where resides.

Mostly we will talk further for mongodb datastore, most of the similar stuff will apply for redis and sqlalchemy.

So we can’t just directly shift queue_controller into ControlDriver from DataDriver, we need to think how to fix queue_controller’s dependencies inside DataDriverBase. Then I realized, the main task is that ‘data plane should not have direct access to QueueController’, rather if data plane want it, it should come thorough control plane. So, just changed the definition of queue_controller() inside DataDriver(and all the places where queue_controller is implemented and directly accessing controllers.QueueController) such that now it call the newly defined mehtod queue_controller of ControlDriver, and this newly defined queue_controller method of ControlDriver will do the tasks that queue_controller of DataDriver was doing; so the code snippet are as follow:

Inside, before any change:

def queue_controller(self):
    return controllers.QueueController(self)

and now afater modification:
def queue_controller(self):
    return ControlDriver(self.conf, self.cache).queue_controller

and newly added queue_controller() inside ControlDriver looks as:
def queue_controller(self):
    return controllers.QueueController(self)

Now we need to manage all other penguins which got disturbed due to this change, so here starts the Real Game!

One thing that is easy to catch is now we need an instance of ControlDriver inside DataDriver, for this first I have created a new instance of ControlDriver inside queue_controller() of DataDriver and from it directly calling ControlDriver’s queue_controller(), the code for DataDriver’s queue_controller looks like the code snippet 2.

But later falper87 suggested that probably we need to create this ControlDriver instance outside the DataDriver. Like creating it in the bootstrap step and passing it to the DataDriver is the right thing to do. This will not create new ControlDriver each time whenever DataDriver.queue_controller() call is made. So now queue_controller() of DataDriver looks something like pasted below:

def queue_controller(self):
    return self.control_driver.queue_controller

No doubt this has changed init() of DriverBase, DataDriverBase, DataDriver and other related classes, all the test cases corresponds to same.  But we have managed them successfully.

Then we need queues_database() inside ControlDriver also from DataDriver, as QueueController is dependent on it. Now how I got to know about it is by looking at QueueCotroller’s definition3. So just shifted queues_database() into ControlDriver from DataDriver. Now while running the tests, got loooooooooooooot many errors, and it was expected as their would be many penguins that got disturbed. We have solved many errors, they are now less than half that were actually generated in starting, but still there are many yet to resolve.

The main error that we are facing right now is *’ControlDriver' object has no attribute 'message_controller'*. Okay, so it says that ControlDriver should have message_controller, why it occurred is obvious, somewhere QueueController would need it because before adding queue_controller to ControlDriver this was not an issue. So solution is just put message_controller inside ControlDriver too as it is in DataDriver.

No that will not be logically correct, message_controller is something that should not be in ControlDriver, it is not a part of control plane. So now what to do? Let us see where does QueueController need it – Few methods(like delete() and stats()) of QueueController need message_controller 4.

So now we need the access of message_controller in QueueController. We can’t directly access message_controller without DataDriver object, as again it won’t be logically correct. If we pass a DataDriver object into ControlDriver or QueueController it will be a circular loop and we will be in trap as we already need ControlDriver instance for DataDriver instance.

This is the whole problem whose solution is yet to find.

Playing with QueueController

Two Merge so far…

Up till now I have completed first step of my task/project as an Outreachy(OPW) intern and now I am working for second step. It is an awesome experience, asking stupid questions, loads of learning, writing open source code, making new friends who are far away. Here I am briefing about few of these points:

Keep asking questions

I got to know about a popular phrase “There is No such thing as a Stupid Question”[1]. And this is true! its okay if you feel like you are asking a stupid question, the person on the other side understands our desire of getting knowledge and most of the time, will satisfy with good answers. I think this is the power of Open Source and Free software communities, they are always willing to help and answer your questions. But at the same time, I believe it is always a good practice to google your questions first(GIYF) before asking, this will save their time and thus will allow them to write more code for the betterment of society. And secondly from Google you can easily gather upper level of knowledge, after which if you will ask, the probability of having a detailed discussion with an expert will increase and thus you will learn even more.

Learning new things

While reading Zaqar’s(OpenStack’s component) code, collaborating with OpenStack community, asking stupid questions and writing open source code, I got to know about @property decorator[2] which help in making private variables inside a class. Well explained here[3]. I have learned about a build tool Maven[4] that is used for OpenStack docs, Debugging in python using pdb[5], python Mixins[6] amazing feature about which I still need to learn more, httpie[7]. The best is I got to know various hidden power of git,  git bisect, git rebase[8], git visualization etc. And finally decided to read more about git, to know about its rest other powerful features. Truly, I become a fan of git. It was dam interesting to know about git bisect.

Its really great to know and learn about various common, old yet very powerful and exciting features of tools/software/technology you use daily. Truly said, ‘knowledge is end less’, the deep you go the more you will realize that yet there is a looooooog path to move on!

Tips for future inters:

Don’t be timid or feel shy or scary in asking questions. If you ask a question it makes you look stupid for 5 minutes – but if you don’t ask – you stay stupid for fifty years, so always ask questions in your life[1].


Two Merge so far…

What FOSS Really Mean

FOSS consist two terms Free and Open Source. But are these both terms different? An open source software is always free software. Isn’t it. This question always blow my mind, why we use both the terms if they mean same. So finally decided to search and get answer. Here putting that only.

Let us understand about each of the two terms individually.

Free Software

Free Software does not mean  the software is free in context of Money/Price. Here Free represents the freedom users have, the liberty users have to read, use, copy, modify and share the code and software(both changed and real one). So it is about rights, privileges and not cost. Users are free to control the program the way they want.

Four degrees of freedom that a free software provide can be read from The Free Software Definition.

Open Source

The origin of this term is to avoid the confusion of the term free software but later on it become something with different perspective. Open Source software means the code of software is public and users are free to modify and enhance and distribute it.

But this is the same as Free Software. So what’s the difference? We are again at same place.


Though both the term refer to all most same thing but there is some small difference. Free Software are more powerful while Open Source license are a bit more supportive to commercial areas. Any program made using free software will also be always published under free software license, but this is not the case with open source software. Any open source software which is not free software if modified can be published without open source license. That is why most of the license that are Free Software are also OSI(Open Source Initiative) approved, that is they are also open source license.

We should not use both terms interchangeably, there are examples of licenses which are only any one type approved and not other. CECILL License v2 is FSF approved while not open source and NASA Open Source Agreement is open source while not Free.

But the real difference between both the term is still a mystery! I will add more stuff with proves if I will find.


What FOSS Really Mean

Switch on to OpenStack :)

For the coming few months I will be working as an OPW intern for OpenStack’s Zaqar component under Flavio Percoco, (flaper87 on IRC). My task is to split control and data planes of Zaqar storage layer completely, brief explanation here.

Exciting..Thrilling..  Exhilarating!.😀


Results are out and internship period is about to start(from 9 Dec), but before this I was an applicant and here sharing experience of that. Yes experience of being an applicant for OpenStack – because it is awesome in itself. I started to work for OPW about a month ago, in October starting and since then I am hacking for it. First I choose 3 organizations whose project ideas were intuitive to me and I was comfortable with the technology stack required, then digging more about different projects for those organizations and interacting with communities, I decided to contribute for OpenStack. (Participants are free to apply for more than one organization.)

Initial Contribution

To turn into an intern from a participant, one needs to submit application(s); and to submit an application participants are suppose to first make some initial contribution for the organization they want to apply. This is really mesmerizing, I never thought I will be able to do that, it sounded very difficult and hard to achieve. Like fixing a bug for such a big OSS, it won’t be easy. That is what I thought earlier. Then I said to myself – “*heh wake up!!…… if you will stop before trying, then it will surely result in failure!*”. I followed all the instructions and advices mentioned on the OpenStack OPW page, installed OpenStack, choose a bug that I found I could work for and started submitting patches for it . In OpenStack such bugs are tagged as ‘low-hanging-fruits’. Bugs with this tag are left by developers only for newbies, so that they can start easily and get familiar with code, patch submission and reviewing system of OpenStack. No doubt OpenStack community was there to help me at all the points where I stuck. I got chance to interact with many former OPW and GSOC interns at IRC channel `#openstack-opw`, with the help of whom I submitted patches. WoooW!!! 🙂 its a victory! I did something which I thought difficult and really hard to do.😀

Keep digging

After submitting one or two patches even they didn’t merged, I started interacting more with mentors, to understand and learn more about project and how to proceed further for it. OpenStack community is really awesome, they welcome each new contributor with open arms🙂. Here putting little talk between me and flaper87 which really inspired me and let me to work more:

    > flaper87 exploreshaifali: please, do keep digging. The more questions you ask,
       the clearer it will be for you :)
    > exploreshaifali flaper87, sure :)
    > exploreshaifali Thanks!!
    > flaper87 exploreshaifali: thank *you* ;)
    > exploreshaifali flaper87, thanking me for what?
    >flaper87 exploreshaifali: for your time, interest and perseverance. I
      appreciate you willing to work on this and I thank you for that. I
      look forward to see you around long enough to do way more.

Wow!! I was really stunned, the Mentor is thanking me :-o . And the last line *”I look forward to see you around long enough to do way more”* is really motivating. It gave me looooots of enthusiasm, zeal and increased my passion and devotion to explore more, work more, learn more.

Submit Application

Discussions with mentor, reading the code of project let me to learn more about project and help in proposing a solution and timeline of the projects for which I was applying. I took help from mentors for this, requested them to review my solutions so that I can make changes if needed, which they did with open arms even being very busy in OpenStack summit and rest of their other stuffs. Meanwhile I started writing application putting all my experience with FOSS and OpenStack both as a user and as a developer, about my past projects and rest all as in the application template; and submit it.🙂

Result Time

Hmmm!!!….., Now its time of result. On November 12 UTC 7 result was out and posted here. The down I scroll in the page the faster my heart beats…. scroll down…heart beats faster….page – down down … heart beats – faster and faster. STOP its OpenStack there…… little more down….woohoooo!!!🙂

LOOOOL!!😀 I did nothing like that😉. I just did `Cnt +f` and typed `Shaifali Agrawal` and yuppppiiieeeee!!!! I could see my name there under OpenStack list without completing my name’s spelling. Haah!!!..

Exciting… Thrilling…  Exhilarating!!…😀

After results are out, I went to my mentor flaper87 and he asked me to first read Mentoring others and yourself. Stunned!!! Motivated looooot more after reading his blog post.🙂 Also he mentioned that talk in his blog too which I have mentioned above, it is extremely  impelling to me. Wow!! It really feels *Super Awesome* to have an opportunity to work under such an exquisite mentor. He is *rocking* indeed and full of fun, helping. Apart from all this he loves mentioning others🙂.

I am looking forward to work hard, learn a lot and ready to bear all ups and downs that will come across the internship period. No doubt **This is just a Start!**

TIP for future OPW Interns :

There is of around three weeks time between application submission last date and result announcement. During that period please don’t sit still. I kept lurking on IRC, interacting with developers there and tried to fix another bug because I knew the more I will dig in it the more I will learn. And hey I think you too don’t want to lose three weeks of learning from experts😉.

What I learned till the result announced is to be patient. I am someone who gets excited very soon and also get panic very soon. But the phase of being an applicant for OPW made me to wait and keep patience if things doesn’t work.  Don’t forget – Things will work if you work!😉

I will take you with me in the whole journey of being an OPW Intern. Stay tuned!!

Switch on to OpenStack :)