Experience

Question 1

Can you tell me more about “Sentiment Analysis and communication strategy optimization for AI powered products”

Answer 1

ChatGPT was just launched and it’s the first time in history that such a high proportion of people had a first hand interaction with a conversational AI, could ask it any question and see how amazing it is

since we havent observed a similar situation before, we didn’t know how this would change people’s perception of tesla’s AI based products. Previously a lot of people were mistrusting AP/FSD and we wondered if they feel differently now as they have more experience with AI.

I needed to collect data but I couldn’t get data from Twitter because it prohibits scapping and the paid subscription would cost me 5k per month

So I found another big forum of Tesla fans and owners and decided to scrap the data from there.

Web Scraper
I built a web scrapper in Python using BeautifulSoup and Selenium packages.
Selenium: used for automating browser interaction
BeautifulSoup: used for parsing HTML and extracting the required data from web pages

* Used Selenium WebDriver to launch a browser (Chrome) and navigate to the forum 
* Used Selenium to scroll, click on the “Next” button, and open each discussion
* I previously found the names of each block on the page by using “Inspect the page” button 
* So I used Selenium methodos like find_element_by_xpath, find_element_by_class_name etc
* Selenium page_Source captured the HTML code of the page and the data is then extracted
* It loos through pages and stores the data in a list of dictionaries which are later converted to a DataFrame 
* So I used a lot of loops and incorporated error handling like try and except blocks 

Data Processing:
* Lower case, remove spaces
* Split into AI related and non AI related
* Cut out the quote of the original messages in the replies
* Remove stop words
* lemmatize (back to the original root rather than stemming – for more accuracy)

Sentiment Classification
Vader and roBERTa

Vader: Lexicon and rule based analysis tool
* Evaluates each word in the text and compounds the score
* Great for social media data
* Used standard threshold because adjusting didn gain results
*
roBERTa (pre-trained Twitter-roBERTa-base-sentiment)
* Bidirectional Encoder representations from Transformers
* Has general knowledge from pre-training, can analyze forward and backward
* Transformer: turns words into a numerical form that a computer can understand (embedding)

Accuracy evaluation:
* Manually labeled a random sample of 97 blog posts
* Used Vader and roBERTa on this sample and evaluated their performance
* Accuracy, Precision, Recall, F1 score
* Accuracy = correct/total
* Precision, how many positive predictions are actually correct = True Positive / (True Positive + False Positive)
* Recall (Sensitivity), how many true positives were correctly identified = True Positive / (True Positives + False Negatives)
* F1, a single score that balances Precision and recall
* Roberta outperformed Vader in all scores

Accuracy: predicting weather
Precision: Spam or Fraud detection (don’t want false positives)
Recall: Disease Screaming or Intrusion detection (rather have false positives than false negatives)

Regression: Difference-in-Difference approach
* Linear regression (linear probability – OLS that uses a binary var as a dependent var)
* Dependent variables: Binary_negative, binary_positive, sentiment_roberta
* Independent (interaction) dummy variables TreatmentControl vs Months
* Dummy variables for months, grouped
binary_negative = β0 + β1 * TreatmentControl + β3 * TreatmentControl * MarAprMay22 + β4 * TreatmentControl * JunJulAug22 + β5 * TreatmentControl * SepOctNov22 + β6 * TreatmentControl * DecJanFeb23 + β7 * TreatmentControl * MarAprMay23 + ε

Parallel trend assumption: in the absence of treatment, the treatment and control group would follow the same trend over time. If coefficients of the interaction terms of pre-treatment are significant (differed), then it’s violated.

Result:
after ChatGPT:
- AI related: positivity decreased and negativity increased, overall sentiment decreased
- Non ai related: positivity increased, negativity is the same

Question 2

Can you tell me more about your experience at Greenchoice?

Answer 2

• Greenchoice ran a campaign to improve customer retention and they needed to evaluate its effectiveness + suggestions for future strategy
  
  • Used propensity score matching to compare customers who received the treatment with those who didn’t to ensure that these groups were similarly
  • the goal was to isolate the effect of treatment on churn
  • determined the effect of the treatment and suggested which segments of customers to target for better results
  • Propensity Score: a probability that a unit would receive a treatment: a logistic regression with a dependent variable treatment/control and independent variables – customer characteristics
  • PSM helps estimate the causal effect of a treatment by accounting for confounding variables, allowing to make more accurate inferences about the effect of the treatment

Question 3

Can you tell me more about your experience at SkinSafe?

Answer 3

SkinSafe is a platform where users can compare different cosmetics products and filter products that contain certain ingredients. Users can check a product and see a safety score. SkinSafe wanted to improve their safety score.
SkinSafe was supposed to share structured data on allergy tests and ingredients for further analysis and optimization. However, they only provided unstructured data based on doctor’s notes about their patients

The task was to analyze it and use this data to improve the product scoring accuracy so that products with a score of 100 are actually safe for most people.

Lots of data processing:
* researching the formal and informal names of chemical components
* extracting the components from the text considering the context
* Structured the data
* Used structured data to evaluate the current product scoring accuracy based on the currently included ingredients
* Suggested to adjust the scoring system to replace some ingredients based on the findings
* Gave gender and age specific recommendations as well
* Doubled the accuracy of the product scoring vs safety

Output:
* a Python code where new data can be plugged, cleaned and categorized
* a dashboard where cleaned data can be plugged and visualize