What I learned about data science through my career

Data science is one such job whose requirements are not well defined. The same profile can have varying requirements depending upon the work item you’re coupled with. Sometimes you have to pull up an excel sheet to present some ad-hoc analysis or spend days engineering a proper solution to help your client achieve their business goals or maintain your production pipeline, etc. Well then what are these mysterious wizards using their sorcery with the data!? Who are data scientists?

Let’s take a step back from our ‘normal’ state and observe; observe…

A Digital detox from your online social life

Image source: Canva

“Tired of it!”, I sighed, while conveniently scrolling through my ‘personalised’ feed as my eyes scanned the content presented on a Full HD screen. I closed the app and thought why I was doing what I was doing. Did I really wanted to feast my mind with average quality barely useful moderately entertaining mind numbing content that provided me almost nothing but nothingness?

“Socialisation, you need to keep in touch with people. News, you gotta stay updated with what’s happening around. …

Understanding the NATURE of a system, the shades of grey.

Image source: Lev Kalmykov — Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=44834594

Let’ say I give you a calculator and ask you to carry out a simple addition; now I’m sure you must be knowing how addition works but if were to ask you how does the calculator carries out that mathematical operation, the chances are that you might say “It just does”. To you the calculator is a magical box that gives you an answer when asked a question; but some of the people must be knowing how the calculator works and won’t step back in giving a detailed working of…

Understanding THE System before understanding A System.

Sometimes it is the very obviousness that hides the simplicity in a seemingly complex world.

To guide you along with the following though exercise, I want you to imagine rainfall or simply put The Water Cycle; yes, the one we studied in our childhood.

Here’s a question for you, how would you know that you have understood something clearly and that you’re confident enough to teach it to others? Well you’re your own judge but consider this:

Einstein, having a final discussion with de Broglie on the platform of the Gare du Nord in Paris, whence they had travelled from Brussels to attend the Fresnel centenary celebrations, said “that all physical theories, their mathematical expressions apart ought to lend themselves to so simple a description ‘that even a child could understand them.’ ~ Einstein: His Life and Times (1972)

“You know, I couldn’t do…

Part 2 of 2: Model Evaluation metrics and Support Vector Machines(SVM)

Who won the race?

Well, the banner image of the blog reveals that it’s the Liner SVC that performed the best on classifying the car acceptability. In case you’re wondering what I am talking about, this blog post is a continuation of one of my previous posts, The Classifier Part(1/2). So after data analysis and pre-processing, here are the results of different models on the data :

One month ago, this day I was sitting at the airport, waiting for my flight back home while reminiscing about all the events, experiences and phases of this ZS Data Science Challenge.

Phase 1: The Online Data Science Challenge

I had completed my third year and it was my summer break. Since the placement season was around the corner, I was honing my coding skills at interviewbit.com. It was early July when I saw that ZS is organizing a Data Science Challenge through interviewbit where top performers will be given PPOs, and of course some amazing prizes to top 3 in first and second rounds. …

Understanding the metrics used for classification

Before posting the second part of my previous project, The Classifier I thought I should write about the metrics used for Classification and why a model can possibly be useless despite having 99% accuracy. Now, what is this accuracy fallacy and what metrics should be used to evaluate the model? Before I answer these questions, let’s try exploring the data. Here, I’ve used a classic example where we have preprocessed data of credit card transactions. The dataset has no null values and is feature scaled already. So I just extracted the useful features from the data after plotting a heatmap.

Part 1 of 2: Data Exploration and Preprocessing

As my quest to explore and understand data continues, I used a dataset from UCI Machine Learning Repository to classify what makes a car acceptable given a few associated features. A crucial part of Data Science is understanding the data. Even in Machine Learning, we need a clean and well-defined data to train our models. So I usually spend a significant amount of time analyzing and/or visualizing the data. I’ve divided this project into two parts, one is for data analysis and the other for model selection. Alright then, let’s explore the data.

After importing the necessary libraries and loading…

Getting Started with “Hello World” of Machine Learning

Linear Regression is basically used to predict the continuous-valued output. In other words, we can predict a dependent variable like laziness using an independent variable such as hours spent watching TV series. After finding the correlation among the variables, we just simply plot a regressor line that best fits our data points. Multiple Linear Regression(MLR) is a Linear Regression with two or more independent variables.

Harjot Pahwa

Problem solver by nature; Data Scientist by profession| ZSer| Ex-JIITian

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store