Multimodal AI Researcher

AI/ML Researcher • Bengaluru, India

Engineering Intelligence.
Unifying Text & Vision.

I am Ayush Mishra, an Artificial Intelligence researcher and computer scientist based in Bengaluru. My research pioneers Cross-Modal Gated Fusion for multimodal misinformation detection, leveraging state-of-the-art Vision Transformers (Swin, CLIP-ViT) and NLP models (DeBERTa, BERT).

Explore Research Architecture View IC2NS2 2026 Paper GitHub LinkedIn

96.66%

Peak Accuracy (SPECTRA)

Multimodal Fake News Detection

IC2NS2 '26

Accepted Conference Paper

BERT & Vision Transformers

8.42 CGPA

M.Tech Data Science

SVNIT Surat (2024–2026)

GATE 514

CS Qualified (2024)

Amazon ML '25 Participant • 250+ DSA

Flagship Research

Cross-Modal Gated Fusion for Fake News Detection

Bridging text semantics and visual context to combat sophisticated digital misinformation through adaptive cross-attention and gated representation learning.

Architecture Flow

Deep Cross-Modal Pipeline

Cross-Modal Gated Fusion Architecture Diagram

Overview

Unified Multimodal Misinformation Framework

Real-world fake news leverages both misleading imagery and biased linguistic phrasing. Our framework extracts deep contextual embeddings from text using DeBERTa/RoBERTa and visual feature hierarchies with Swin Transformer & CLIP-ViT.

Text Encoders: DeBERTa-v3, RoBERTa-large, BERT

Vision Encoders: Swin-B (Shifted Windows), CLIP-ViT-B/16

Fusion Mechanism: Learned Gated Unit \( g = \sigma(W_g \cdot [h_t; h_v]) \)

Loss Objective: Cross-Entropy with Label Smoothing

Fakeddit Benchmark Dataset

Multimodal Reddit dataset with fine-grained misinformation labels

Large Scale

91.15%

Classification Accuracy

89.78%

Macro F1 Score

SPECTRA Benchmark Dataset

Comprehensive multimodal dataset for social media veracity

Peak SOTA

96.66%

Classification Accuracy

96.75%

Macro F1 Score

Peer-Reviewed Research

Publications & Proceedings

Academic conference papers and scholarly contributions in multimodal artificial intelligence and transformers.

International Conference

IC2NS2 2026 • Accepted & Presented • 2026

A Unified Multimodal Framework for Fake News Detection Using BERT and Vision Transformers

Ayush Mishra, et al.

International Conference on Intelligent Computing, Cognitive Networks, and Smart Systems (IC2NS2 2026)

Abstract: The proliferation of digital misinformation across modern web platforms poses severe societal challenges. In this paper, we propose an integrated multimodal deep learning framework that harnesses the contextual representation power of Bidirectional Encoder Representations from Transformers (BERT) along with hierarchical Vision Transformers (ViT/Swin). By introducing a gated cross-modal fusion layer, our network adaptively weights linguistic credibility indicators against visual incongruities, effectively suppressing deceptive signal propagation. Extensive empirical evaluations validate superior classification performance and generalization across diverse fake news benchmarks.

Architecture Details →

Engineering & Systems

Featured Projects

Live Utility Platform

AllAgeCalculator.com

High-precision chronological and date analytics platform designed with responsive UI, high-speed client-side calculation algorithms, and SEO-optimized architecture delivering fast user insights.

JavaScript Web Algorithms SEO Architecture Modern UI/UX

Visit Live Web App

🔬 Core Thesis Project

Cross-Modal Gated Fusion Framework

PyTorch deep multimodal system fusing DeBERTa/RoBERTa language models with Swin Transformer & CLIP-ViT visual encoders for cross-modal verification. Evaluated on Fakeddit (91.15% Acc) & SPECTRA (96.66% Acc).

PyTorch Hugging Face DeBERTa Swin Transformer CLIP-ViT Python

View Architecture & Metrics →

96.55% Accuracy

Coccidiosis Disease Classification

End-to-end deep learning computer vision system for poultry fecal disease detection. Achieved 96.55% accuracy and 0.12 validation loss with DVC dataset versioning and automated GitHub Actions CI/CD pipelines.

Python CNN DVC GitHub Actions CI/CD

Explore Pipeline →

R² > 90% Production MLOps

Student Math Score Regression & MLOps

Production-grade regression pipeline benchmarking 8 models (XGBoost, CatBoost, Random Forest, GradientBoost). Integrated MLflow & DagsHub for tracking, SHAP for feature explainability, and deployed with Streamlit.

Scikit-learn XGBoost MLflow SHAP Streamlit DagsHub

View Experiment Tracking →

Academic Faculty

Teaching & Academic Experience

Empowering the next generation of engineers with rigorous foundations in Data Science, Machine Learning, and Computer Science.

DSCE

Researcher & Faculty Mentor

Dayananda Sagar College of Engineering (DSCE)

Department of Information Science and Engineering (ISE) Bengaluru, Karnataka, India

Guiding students and research scholars in Machine Learning, Deep Vision architectures, and Data Science. Bridging cutting-edge multimodal intelligence research with practical engineering systems and academic mentorship.

📚

Curriculum Leadership

Delivering comprehensive lectures in Data Science, Machine Learning, and Algorithms.

💡

Undergraduate Research Mentorship

Guiding student capstone projects in multimodal deep learning, computer vision, and NLP.

🔬

Laboratory & Hands-on MLOps

Designing practical pipelines using PyTorch, Hugging Face, Scikit-learn, and Cloud tools.

Core Teaching & Research Interests

Multimodal Deep Learning: Cross-attention, Vision Transformers & Text Encoders
Applied Machine Learning: Classification, Regression & Statistical Modeling
Natural Language Processing: Transformers, BERT, Tokenization & Semantics
Data Engineering & MLOps: Feature Engineering, DVC, CI/CD pipelines
Data Structures & Algorithms: Problem Solving & Optimization

“

"Our teaching philosophy centers on active inquiry: deconstructing complex mathematical concepts into first principles and deploying reproducible code."

— Ayush Mishra, M.Tech (SVNIT)

Core Competencies

Technical Expertise & Stack

Curated toolchains and methodologies applied across research and production environments.

🧠

Multimodal AI & Transformers

Advanced Deep Learning Architecture

📊

Data Science & Math

Analysis & Statistical Modeling

⚙️

MLOps & Infrastructure

Reproducibility & Deployment

💻

Programming Languages

Systems & Scripting

🏆

Problem Solving

Algorithmic Foundations

Academic Pedigree

Education & Qualifications

Foundations from premier national institutes in India.

2024 – 2026

M.Tech in Computer Science and Engineering

CGPA: 8.42 / 10

Specialization in Data Science

Sardar Vallabhbhai National Institute of Technology (SVNIT), Surat

Advanced coursework in Multimodal Deep Learning, Natural Language Processing, Computer Vision, High Performance Computing, and Machine Learning Systems. Completed thesis on Cross-Modal Gated Fusion architectures for misinformation detection.

2020 – 2024

B.Tech in Computer Science and Engineering

CGPA: 7.81 / 10

Deenbandhu Chhoturam University of Science and Technology (DCRUST), Sonipat

Rigorous foundations in Algorithms, Database Systems, Object-Oriented Software Engineering, Operating Systems, Computer Networks, and Artificial Intelligence.

Get in Touch

Let's Connect & Collaborate

Open to research discussions, academic collaborations, guest lectures, and student project inquiries.

Direct Email ayushmishra642001@gmail.com

Phone +91-9992619779

Academic Affiliation

Dayananda Sagar College of Engineering
Dept. of Information Science & Engineering
Shavige Malleshwara Hills, Kumaraswamy Layout, Bengaluru, Karnataka 560078

Send a Message

Looking to discuss research collaborations, academic inquiries, or projects?

Engineering Intelligence. Unifying Text & Vision.