Lately I’ve been messing around with RabbitMQ quite a lot. Basically it is a tool, that can make messaging between services reliable. Key word in the latter is “can”, because it doesn’t have to. As default you probably have transient messages on non-durable queue. Well, it’s allright if you only use it to make consuming those messages “neat”, but most of the time you want to use queue to make your messages persistent. But it’s not as easy as I thought it would be.
Firstly, you need to change all the settings for your queues in all the services that are using it. You can’t just make it durable from within rabbitmq itself, because it would throw an error, that incoming message is for non-durable queue and this particular message belongs to durable one with the same name.
Secondly, durable queue is not the only thing that is needed for messages to be persistent. You have to set them to be persistent, because by default they are transient and if queue service gets to be restarted, transient messages won’t be recovered.
Third and in my opinion toughest thing to be dealt with:
Let’s say that you remembered about making your queue durable and all your messages persistent. What will happen if some corrupted message gets sent to consumer? It will make an infinite loop of errors, because every time restart happens it won’t make the problem disappear. So you need to make sure you have a handler for messages that can’t be acknowledged by consumer.
That’s how I got to know about “Dead letter queue”. After you implement it basically the problem boils down to make your queue aware of this mischievous situation, so it can send corrupted message to the dead letter queue, that exists (afaik) only for debugging purposes. That way your queue gets “unlocked”, and your consumer services are working as a charm. So, if somehow my past self gets to read this blog, just google this one, when you will be thinking about how to make it all working after queue restarts 😉
From the day I started to work as a programmer, I can’t stop thinking about to where I should aim with my career. Should I expand on my web development skills, or maybe go embedded? Maybe I should focus on DevOps/SysOps? What about new technologies like augumented/virtual reality or machine learning?
I know that it is the beginning of my career and I still have time to choose, but being torn between multiple areas of expertise is not really comfortable for me. But everything feels so exciting and also future-proof. There are things to consider though. Am I smart enough to be good at what I choose? Can I find job where I can make my skills shine and grow at the same time? What about my income after I make up my mind?
This is what makes my heart beat faster:
I really like the flexibility and how pragmatic web development is. There are infinite number of possibilities, and there is almost instant gratification aspect – you can make things fast with glamorous effect in the end.
When I was younger I wanted to be inventor, to make life easier for some people, and thinking about augumented/virtual reality brings my childhood dreams back to life again. I got mind full of ideas on how to e.g. help people with mental illness. Who knows, maybe I could even save someone’s life with one of my products.
Going C/C++ route makes my inner linux fan boy cry from joy. It is my long-time dream to make my contribution to linux kernel. Making open-source Internet of Things (IoT) systems is quite appealing too. But only if it makes someone’s life better and independent from big companies, not all those fancy spying machines.
Not so long ago I read an article on machine learning in medicine for diagnose purposes and it was inspiring. Thought about how big of an impact my work would do makes me quiver.
but those are just fantasies, because all in all there are other, more earthbound, factors like number of job offers that I could send application for, income (after all we all need money to live), my education (which is none) and work-life balance.
But you know what? I like to think that maybe someday I will find myself confident enough to recruit myself in one of the companies that are doing one of those things I listed and make my positive impact on the world with my skillset. This is big factor for me when it comes to my dedication for learning programming…
I was thinking a lot about frameworks in general lately. Framework is this set of tools, that helps programmer to just focus on business logic instead of language. What is the difference between framework and library then? I heard on one polish podcast (listen) a nice quote that I’ll paraphrase – “Framework is what calls you, You are the one, who calls a library”. Given this definition one can conclude that often you have to give up your flexibility to speed up programming your application when you choose to pick up framework. There are people much smarter than me, that are arguing about when to use frameworks so I want to focus on the other problem – junior level one. From beginners perspective, when you are learning a programming language it’s really hard to tell when you should start to learn a framework.
Let me give you an example from my recent experience.
Because of one microservice that I had to add functionality into, I had to dive into Ruby language, because it was written in Ruby on Rails (RoR) framework. With only a glimpse of knowledge about the language itself and no prior experience with RoR I had to use websockets (that I knew nothing about) for my feature, so three solutions came to my mind:
– find websockets gem with good documentation and just use it at the same time making my already huge application even bigger
– create websocket functionality with vanilla ruby. To do it I would need to be reeeaaally comfortable with Ruby language (minuses: it will take a long time to make it work and I would go against the rule to not reinvent the wheel)
– use a tool that is inside the framework that the application is already in (Active Cable).
I was asking myself questions. How to dive into it? Should I start with basics of language just to know what I’m looking at and to see the difference between language and its framework, or should I just pick up RoR first and guess language syntax from the usage?
The best approach for me was to just do what you have to do to achieve a goal in a fastest possible way. I researched and planned what I need to code, to make it work, so rather than doing language tutorial or going through full documentation on framework, I just googled how others implemented what I’m trying to do in ruby. Most people used Active Cable, so why I shouldn’t? But hey – It’s not that I didn’t have to learn vanilla Ruby at all, while going through all those framework built-in methods.
I found out that diving into a language with a framework from day one has one big merit – you will do a lot of things the way it is done in commercial projects and as a side effect you will slowly gain confidence in language you’re using, because after all you need to fill the framework with business logic to make application whole.
Ok, as I promised, here is the second part of my ansible journey. Previously (Ansible #1) I suggested that you can rent a server for free to follow my series. I hope you did it, because in this article I assume you already have a running remote machine. I have one with ubuntu distribution inside for the purpose of this article. Remember to install python on it, because ansible is dependent on it (ssh to your remote – you can check aws documentation to see how to do it, and if you have ubuntu type sudo apt-get install python-minimal in your console).
After you installed it you can check if you really did:
Example output (fedora):
config file = /etc/ansible/ansible.cfg
configured module search path = Default w/o overrides
python version = 2.7.13 (default, Jun 26 2017, 10:20:05) [GCC 7.1.1 20170622 (Red Hat 7.1.1-3)]
Now you have the tool for managing your remote host, but we need it to know to which machine it should refer. Ansible have its hosts file, that defaults to /etc/ansible/hosts. Open it with sudo privileges and your favourite text editor. I used vi.
sudo vi /etc/ansible/hosts
Now, open your browser and go to your aws console, onto ec2 tab and copy IPv4 public address of your server and paste it to your hosts file. It should look similar to mine (I x’ed part of my IP for privacy)
Save and exit. And as with install, we can check if everything we did went fine. Let’s ping our remote!
You should already know from aws documentation that you can’t connect to your remote without ssh key. I keep my key inside my_keyname.pem file. We will need it to ping our server. If you too have ubuntu on your remote, we need to become remote ubuntu user from ansible. Full command for ping should look like this (given that you have your terminal open in the same directory you have your .pem file in)
ansible all -m ping -u ubuntu --private-key "my_keyname.pem"
-m method is for module we are using to ping the remote.
-u user is for as which user we are connecting with.
–private-key “filename” is to point ansible where it should look for ssh key to connect, you can use ssh-agent ignore it, but I will leave that for you to check out single-handedly.
If output is green and response includes “pong”, you got the connection! Congratulations!
Now let’s use it for something. As an example I will install a small program called “ctop” (https://github.com/bcicen/ctop). As you can see from its readme page, installation is trivial, so it is a good example to begin with.
Basically we need to do two things to make this work
1. Download ctop
2. Make it executable
Let’s start and create a playbook to provide a task for our host. First let’s create .yml file, e.g. install_ctop.yml. Open it in your favourite editor. Now we need to tell on which hosts we want our tasks to be done, but since we have only one host, we can use “all” argument.
We defined mode with 0555, to make file readable and executable. If you don’t know what mode to pick when you want your file to be executable/readable/writable I suggest you visit this site: GNU/Linux permissions If you don’t know what am I talking about here, google chmod and chown, it will ease your pain in Linux environment by a lot 😉
So at the end our first playbook should look like that
But let’s say you are afraid you messed up the indentation. No problem, just run your playbook with –syntax-check flag
ansible-playbook install_ctop.yml --syntax-check
And if it outputs the filename instead of error, you are the king of yaml!
Ok, so now we know that its syntax is ok, but what if we still are afraid that this task will ruin our host, because we got it all wrong or we maybe mistyped hosts in which it should run. You just need to add –check flag and it will run the playbook without making any changes to remote.
If everything looks fine after all, you can just erase –check flag and make your first ansible dream come true 😉
Remember to ssh to your remote to check if everything went fine, just to cherish the moment, when you feel like a professional DevOps
ctop version 0.6.0, build 617b1b2 (version may vary)
Don’t worry if you tried to run ctop without -v flag and it gave you error, it still means you have installed it – you just need to have docker installed for it to run. But that’s another tool for another story 😉
If something went wrong, contact me, we will figure it out together!
Well, It was fun for me and if you feel the same and you want me to help you configure/install something more complicated next time – let me know!
Note to myself from the past: Don’t let yourself sink in all those threads about which programming language is best to start. It’s a dead end. Just grab one and stick to it for few months at least.
I lost a great amount of time back when I was teenager, because I was doing all kinds of tutorials that brought me to the worst spot. I was thinking “so, with those if statements and for loops you can build a whole application or a game? How? I must have missed something”. So I tried another tutorial with hope that it will bring me closer to reveal the mystery how to build something big with those tiny little bricks.
Now – if I could go back in time I would just pick something I want to make. Let’s say I want to build an audio player. I would open text editor, make a step by step plan and just google everything, but one step at a time. For example:
how to create a window in X language
how to make buttons in window in X language
file picker in X language
how to link buttons with file picker in X language
how to play music with X language
and so on. Of course sometimes you need to rephrase questions multiple times before you find an answer on Stack Overflow or some blog, but you will get there. Just be patient, because you will read a lot more than you write at the beginning. At least that’s what I do most of the time.
And yeah – making each step working is hell of a lot satisfaction and joy. Don’t make errors let you down. Make yourself enjoy it.