Social Media Caption Generator

Description:

This project has the capability of being used by social networking platforms or other image-dependent platforms as a “Automatic Social Media Caption Generator”. Social media platforms like Instagram, Facebook, Twitter, Pinterest are highly dependent on how the image descriptions are set up with a good caption and hashtags. Our model analyzes the objects in the image and generates categories that can be used as “hashtags” and further recommends captions to the user as they upload images into the application.

Technologies Used:

• Predictive Modeling
• Deep Learning
• Image Processing
• MultiClass Text Classification
• Web Scrapping

Environment

Python (NumPy,Pandas,Tkinter,Scikitlearn,NLTK,TensorFlow, Keras),YouTube API

Flow Diagram

Phase 1: Generating Image description
In this phase, we implemented two sequential models using pre-trained models in Keras with Tensorflow backend. The first model is responsible to extract the features from images for which we are using the Residual Network Model. The second model is for creating word embeddings after performing Natural Language processing on the image descriptions in the trained dataset.The Flicker30k dataset was used in order to train the two sequential models.

Phase 2: Predicting the Category of the Image Description
In this phase, we used the output from phase 1 to predict the category of the image to classify the image description into 9 different categories adventure, art and music, food, history, manufacturing, nature, science and technology,sports,travel. As the labelled dataset wasn’t openly available , a labelled dataset was created by collecting the data using Youtube API and labelling the youtube video descriptions based on different queries.

Phase 3: Building the UI
In this phase, a UI was built using TKinter to run the models that were built in phases 1 and 2. Post development of the UI, an executable file was created which can be used by anyone to run this application easily.

DEMO EXECUTION

Click below on the Video