Receiving messages from Amazon SQS in a Spring Boot applicationTech by Sunny Srinidhi - January 16, 2020January 24, 20200 In this post, we'll see how we can receive messages from an Amazon SQS queue in a Spring Boot application. This is a continuation of the previous post where we talked about how we can send messages to an SQS queue. The obvious next part of that is how do we receive those messages. So in this post, we'll do just that. If you don't have an Amazon SQS queue created already, checkout the previous post on how to do it. Here, I'll assume that you already have that pipeline setup. So I'm going to skip that part of the post. We'll jump right into the code. The Code The first thing we need to add in our Spring Boot application is the
Sending messages to Amazon SQS from a Spring Boot applicationTech by Sunny Srinidhi - January 14, 2020January 24, 20203 We're looking at yet another proof of concept (POC) application today. We're going to see how we can integrate Amazon SQS, which stands for Simple Queue Service into our Spring Boot application so that we can send messages to the queue. I'm going to use a few terms in this post which are influenced by Apache Kafka, because I come with extensive Kafka experience. However, I'm not going to compare Apache Kafka and Amazon SQS here. To clear things up, any service which sends a message to an SQS queue, I'll refer to such a service as the producer. And any service which receives a message from an SQS queue, I'll refer to that as the consumer. Now that we
Proof of Concepts (POCs) I write a lot of POC projects, especially when I'm learning something new or I need to quickly test if a data pipeline works, or maybe I'm just testing a new integration. I make all these POCs public as Github repositories. I wanted to consolidate the list of POCs in an easy to search fashion. And that's why I have this page here. Below is a list of all the POCs that I've written so far. If a particular POC has an accompanying blog post which explains the code in the POC, I have linked that blog post as well in the list below. Let me know if any of these POCs have helped you in any way.
Use Apache Drill with Spring Boot or Java to query data using SQL queriesData ScienceTech by Sunny Srinidhi - September 24, 2019April 7, 20200 In the last few posts, we saw how to connect Apache Drill with MongoDB and also how we can connect it to Kafka to query data using simple SQL queries. But when you want to move this to an actual real world project, you can't sit around querying data from a terminal all day long. You want to write a piece of code which does the dirty work for you. But how exactly do you use Apache Drill within your code? Today, we'll see how we can achieve this with Spring Boot, or pretty much any other Java program. The Dependencies For this POC, I'm going to write a simple Spring Boot CommandLineRunner program. But you can use pretty much any other
Apache Drill vs. Apache Spark – Which SQL query engine is better for you?Data ScienceTech by Sunny Srinidhi - September 23, 2019February 13, 20200 If you are in the big data or data science or BI space, you might have heard about Apache Spark. A few of you might have also heard about Apache Drill, and a tiny bit of you might have actually worked with it. I discovered Apache Drill very recently. But since then, I've come to like what it has to offer. But the first thing that I wondered when I glanced over the capabilities of Apache Drill was, how is this different from Apache Spark? Can I use the two interchangeably? I did some research and found the answers. Here, I'm going to answer these questions for myself and maybe for you guys too. It is very important to understand that
Getting Started with Apache Drill and MongoDBData ScienceTech by Sunny Srinidhi - September 23, 2019February 28, 20203 Not a lot of people have heard of Apache Drill. That is because Drill caters to very specific use cases, it's very niche. But when used, it can make significant differences to the way you interact with data. First, let's see what Apache Drill is, and then how we can connect our MongoDB data source to Drill and easily query data. What is Apache Drill? According to their website, Apache Drill is "Schema-free SQL Query Engine for Hadoop, NoSQL and Cloud Storage." That's pretty much self-explanatory. So, Drill is a tool to query Hadoop, MongoDB, and other NoSQL databases. You can write simple SQL queries that run on the data stored in other databases, and you get the result in a row-column format. The
About Me Connect with me on: Twitter | LinkedIn | Medium Products Links Links is a simple bookmarking service which allows you to bookmark your favorite websites from your Android device, or from the Chrome browser. The service also lets your organise your bookmarks into various folders so that its easy to keep track of your bookmarks. Your bookmarks are synced between your Chrome browser and your Android device. So no matter if you're on a desktop, a laptop, an Android smartphone, or an Android tablet, your bookmarks are available. You can have a look at the web interface and register, which will let you use the Chrome extension and the Android app. Nothing Pro As the name suggests, this app does absolutely nothing. It just has a label which says, well,