Optimising Hive Queries with Tez Query Engine Data Science by Sunny Srinidhi - June 13, 2022June 13, 20220 Hive provides us the option of executing SQL queries with a few different query engines. It ships with the native MapReduce engine. But we can switch that to Tez which has gained popularity since its launch, or we can also use Apache Spark as well. Read more... “Optimising Hive Queries with Tez Query Engine”
Understanding Apache Hive LLAP Data Science by Sunny Srinidhi - November 18, 2021November 18, 20210 Apache Hive is a complex system when you look at it, but once you go looking for more info, it’s more interesting than complex. There are multiple query engines available for Hive, and then there’s LLAP on top of the query engines to make real-time, interactive queries more workable. Read more... “Understanding Apache Hive LLAP”
Installing Hadoop on the new M1 Pro and M1 Max MacBook Pro Data Science by Sunny Srinidhi - November 5, 2021November 5, 20213 In the previous series of posts, I wrote about how to install the complete Hadoop stack on Windows 11 using WSL 2. And now that the new MacBook Pro laptops are available with the brand new M1 Pro and M1 Max SOCs, here’s a guide on how to install the same Hadoop stack on these laptops. Read more... “Installing Hadoop on the new M1 Pro and M1 Max MacBook Pro”
Installing Zsh and Oh-my-zsh on Windows 11 with WSL2 Tech by Sunny Srinidhi - October 27, 2021October 27, 20211 Before we begin, you might ask, why am I writing on something this trivial? I sold off my old MacBook Pro because I’m super excited about the new M1 Pro MacBook Pros. I have pre-ordered one of those and am waiting for it to come. Read more... “Installing Zsh and Oh-my-zsh on Windows 11 with WSL2”
Getting Started With Apache Airflow Data Science by Sunny Srinidhi - October 11, 2021October 11, 20210 Apache Airflow is another awesome tool that I discovered just recently. Just a couple of months after discovering it, I can’t imagine not using it now. It’s reliable, configurable, and dynamic. Because it’s all driven by code, you can version control it too. Read more... “Getting Started With Apache Airflow”
Fake (almost) everything with Faker Data Science by Sunny Srinidhi - September 30, 2021September 30, 20210 I was recently tasked with creating some random customer data, with names, phone numbers, addresses, and the usual other stuff. At first, I thought I’ll just generate random strings and numbers (some gibberish) and call it a day. But then I remembered my colleagues using a package for that. Read more... “Fake (almost) everything with Faker”
Querying Hive Tables From a Spring Boot App Data Science Tech by Sunny Srinidhi - June 30, 2021June 30, 20211 In this post, we’ll see how we can query tables that reside in Hive using a Spring Boot application. As always, I’m going to use a Spring Boot web app with a few GET APIs to show how we can query data from Hive. Read more... “Querying Hive Tables From a Spring Boot App”
Lemmatization in Natural Language Processing (NLP) and Machine Learning Data Science by Sunny Srinidhi - February 26, 2020February 26, 20200 Lemmatization is one of the most common text pre-processing techniques used in Natural Language Processing (NLP) and machine learning in general. If you've already read my post about stemming of words in NLP, you'll already know that lemmatization is not that much different. Both in stemming and in lemmatization, we try to reduce a given word to its root word. The root word is called a stem in the stemming process, and it is called a lemma in the lemmatization process. But there are a few more differences to the two than that. Let's see what those are. How is Lemmatization different from Stemming In stemming, a part of the word is just chopped off at the tail end to arrive at
Stemming of words in Natural Language Processing, what is it? Data Science by Sunny Srinidhi - February 19, 20201 Stemming is one of the most common data pre-processing operations we do in almost all Natural Language Processing (NLP) projects. If you're new to this space, it is possible that you don't exactly know what this is even though you have come across this word. You might also be confused between stemming and lemmatization, which are two similar operations. In this post, we'll see what exactly is stemming, with a few examples here and there. I hope I'll be able to explain this process in simple words for you. Stemming To put simply, stemming is the process of removing a part of a word, or reducing a word to its stem or root. This might not necessarily mean we're reducing a word
An Intro to Affective Computing Data Science by Sunny Srinidhi - January 7, 2020January 7, 20200 Not a lot of us have heard of Affective Computing. Most people I have spoken to about this didn't know anything about Affective Computing. So I thought, I'll just write an intro, explaining what I have understood about the discipline and hopefully, will get to learn more from the comments. So let's get started. Affecting computing is all about understanding human emotions in a human-machine interface system and responding based on those emotions. Consider this, you get into an ATM vestibule to draw some cash, but you're tensed about getting late to your date, who is already waiting for you at the restaurant. If anybody sees you in this condition at the ATM vestibule, they'll be able to easily understand that