Rasa NLU: Comparison of versions

Here is a comparison of the performance of Rasa NLU version 0.12.2 on Python 2 with Rasa NLU version 0.13.8 on Python 3. As you may see this is a slightly different comparison than what people would normally do - I happened to compare different versions of Rasa NLU on different versions of Python. This difference in performance is something I found out while I was trying to move the Telegram web hook of my chatbot from a web app on Python 2 to a web app on Python 3. I saw a degraded performance on the 0.13.8 version on Python 3 when compared with 0.12.2 version on Python 2.

With 0.12.2 version on Python 2, the Rasa interpreter took 30 seconds to load for the first time, so the first response took upto 30 seconds. All further responses were within 2 seconds. This was a decent response time given my chatbot is not used heavily yet. When I happened to move the Telegram webhook (and the code) to the app on Python 3, the interpreter was taking 30 seconds for every request. This was definitely not an acceptable time, not even to me, leave the users.

For reference, both my apps are hosted on Pythonanywhere, a PaaS platform for developing and deploying Python apps. Pythonanywhere by itself is built on top of AWS, an IaaS platform. I have 3 GB of RAM allocated to my paid account.

The whole reason why I moved to the website on Python 3 was to remove my other old website. I apparently had to move migrate my Python 3 website to Python 2 for my chatbot to continue working fine while I took off the old website.

Hope this info will help people as a heads to properly test if they are looking to upgrade Rasa and Python versions.

15:24 ~ $ pip freeze –user | grep rasa

rasa-nlu==0.12.2

15:24 ~ $ pip3 freeze –user | grep rasa

rasa-nlu==0.13.8

PS: This a comparison I found 1 month back, never got a chance to put it in a blog till today.

About the author

Vishnu Vardhan Chikoti is a co-author for the book "Hands-on Site Reliability Engineering". He is a technology leader with diverse experience in the areas of Application and Database design and development, Micro-services & Micro-frontends, DevOps, Site Reliability Engineering and Machine Learning.