Archives

Data Science

Backward Elimination for Feature Selection in Machine Learning

Sunny SrinidhiNovember 11, 201967 Views0

When we're building a machine learning model, it is very important that we select only those features or predictors which are necessary. Suppose we have 100 features or predictors in our dataset. That doesn't necessarily mean that we need to have all 100 features in our model. This is because not all 100 features will have significant influence on the model. But then again, this doesn't mean it will be true for all cases. It depends entirely on the data we have in hand. Here is more info about why we need feature selection. There are various ways in which you can find out which features have very less impact on the model and which ones you can remove from your dataset. I have written about feature selection before, but that was very brief. In this post, we'll look at Backward Eliminati...

Read More
Tech

Sub-6 and Millimeter Wave (mmWave) frequencies for 5G – All you need to know

Sunny SrinidhiNovember 9, 201969 Views0

5G is the next obvious upgrade to 4G and LTE that we use extensively today for our data needs when we're on the go. LTE was a huge upgrade from the much slower 3G a few years back. But in 2019, we're seeing over 1Gbps speeds with 5G. To make this a reality, wireless carriers are using a combination of different technologies and waves. In this post, I'll try to explain two of those which we see and hear in most conversations revolving around 5G - Sub-6 and Millimeter waves. To understand sub-6 and millimeter waves (mmWaves), we first need to understand how our smartphone radio signals function. As you all know, we have cell phone towers or antennas placed all around us which provide our phones with the coverage they need to keep us connected to the network. But what most people don't kn...

Read More
statistics-on-laptop
Data Science

Null Hypothesis and the P-Value

Sunny SrinidhiNovember 8, 201997 Views1

When you're starting your machine learning journey, you'll come across null hypothesis and the p-value. At a certain point in your journey, it becomes quite important to know what these mean to make meaningful decisions while designing your machine learning models. So in this post, I'll try to explain what these two things mean, and you try to understand that. Now, if you don't have a background in statistics, the definitions of null hypothesis and p-value will make no sense to you. It's just gibberish going way over your head. That's what happened to me the first few times I tried to understand them. It took me a good couple of days to get an idea of what they mean. I could still be wrong in my understanding to this very day. And I'm sure that you guys will have more knowledge about t...

Read More
encryption
Tech

How to encrypt a string in Java using RSA and decrypt it in Python

Sunny SrinidhiNovember 7, 2019128 Views0

Recently at work, I was tasked to write a Java program which would encrypt a sensitive string using the RSA encryption algorithm. The encrypted string would then be passed on to a client over public internet. The client would then use the private key to decrypt the message. But the client is written in Python. So I have to make sure the encryption and decryption wok as expected. And as always, I wrote POCs for both. And here, I'm going to document that. Creating the key pair Before we can start the encryption, we need to have a key pair. A key pair will have a public key and a private key. The public key, as the name suggests, is public. You can share it with anybody who wishes to send you an encrypted text. They will encrypt the original text using this public key, and send over th...

Read More
scikit_learn_logo
Data Science

Fit vs. Transform in SciKit libraries for Machine Learning

Sunny SrinidhiNovember 7, 2019253 Views0

We have seen methods such as fit(), transform(), and fit_transform() in a lot of SciKit's libraries. And almost all tutorials, including the ones I've written, only tell you to just use one of these methods. The obvious question that arises here is, what do those methods mean? What do you mean by fit something and transform something? The transform() method makes some sense, it just transforms the data, but what about fit()? In this post, we'll try to understand the difference between the two. To better understand the meaning of these methods, we'll take the Imputer class as an example, because the Imputer class has these methods. But before we get started, keep in mind that fitting something like an imputer is different from fitting a whole model. You use an Imputer to handle missi...

Read More
MachineLearning
Data Science

ColumnTransformer in SciKit for LabelEncoding and OneHotEncoding in Machine Learning

Sunny SrinidhiNovember 6, 2019438 Views1

In a very old post - Label Encoder vs. One Hot Encoder in Machine Learning - I had demonstrated how to use label encoding and one hot encoding to separate out categorical text data into numbers and different columns. But the SciKit library has come a long way since I wrote that post, and it has made life a lot more easier. The developers of the library might have realised that people use LabelEncoding and OneHotEncoding very frequently. So they decided to come up with a new library called the ColumnTransformer, which will basically combine LabelEncoding and OneHotEncoding into just one line of code. And the result is exactly the same. In this post, we'll quickly take a look at how we can do that with some code snippets. The Code First, as usual, we need to import the required li...

Read More
Tech

SpaceX shares video of successful parachute tests of the Crew Dragon spacecraft

Sunny SrinidhiNovember 4, 201973 Views0

SpaceX today shared of video on Twitter of it testing the parachutes for the Crew Dragon spacecraft. And this test was the 13th successful test of the parachutes, in a row. SpaceX only had a target of 10 successful tests by the year end, as Elon Musk had stated in a joint press conference with NASA last month at SpaceX headquarters in Hawthorne, California. So its a new achievement. https://twitter.com/SpaceX/status/1191067348914098176?s=20 In this latest test video, the company intentionally did not deploy one of four parachutes, just to demonstrate that it is safe even it of them fails to deploy. This is a rare occasion for SpaceX, because this time they are actually ahead of the schedule that Elon Musk had set for the testing of these third generation parachutes. Talking of...

Read More
Data ScienceTech

Invoke an AWS Lambda Function from another Lambda Function

Sunny SrinidhiNovember 4, 2019186 Views0

I recently discovered that you can't invoke more than one Lambda function in AWS for an S3 event, with the same prefix and suffix (or just with the same suffix, which was the issue in my case). So I wanted a way to invoke one Lambda function from another Lambda function. If you're feeling kind of lost, check out the problem statement in my Github project. That could possibly add some context to the problem. If you don't want to go there, I'll try to explain it here again. The Problem and the Requirement In one of our projects, we have a Lambda function which is invoked whenever a text file is uploaded to a particular S3 bucket. The Lambda function takes that file, does some processing on the data in that file, and then does something else which I don't really remember anymore. Last ...

Read More
Bixby Routines
SmartphonesTech

Bixby Routines, they actually work!

Sunny SrinidhiNovember 2, 201994 Views0

If you, for some reason have been living under a rock and don't know what Bixby is, it's the virtual assistant that Samsung has been trying to shove down your throat for a while now. But fortunately, with their latest smartphones, the Galaxy Note 10 series, they've given the option to silence Bixby forever, with the "Side Key" option. Today, we're not going to talk about how horrible or awesome the virtual assistant is, rather, how some features of Bixby are actually very useful, and work as expected. We're going to talk about Bixby Routines. On my Galaxy Note 9, I was using a third party app to map the Bixby key to open up the Google app, and had mapped Bixby to the double press of the key. But with my new Galaxy Note 10 Plus, I have the option to completely remove Bixby integration w...

Read More
Tech

How to automatically trigger AWS Lambda functions using CloudWatch

Sunny SrinidhiNovember 2, 2019139 Views0

If you have AWS Lambda functions which need to be triggered periodically, like CRON jobs, there are many ways to achieve this. But I recently discovered a very easy and AWS-way of doing this, which makes life a lot easier. So, there are a lot of ways you can trigger Lambda functions periodically. One of the most common ways I've see people doing this is adding an API Gateway to the Lambda function, and then calling that API periodically as a CRON job from one of the machines in the setup. I actually thought this is how you're supposed do to that. Okay, let me make this clear. I'm not a DevOps guy. I just learn these things as and when the job requires me to. So the CloudWatch feature that I recently discovered is by no means a new feature introduced recently. It just so happens that I ...

Read More