# Backend MVP Tasks - AI Study Assistant > ⚠️ **IMPORTANTE**: Este documento foi atualizado para refletir a arquitetura REAL do projeto. > > **NÃO EXISTE BACKEND NODE.JS/TYPESCRIPT**. O projeto utiliza apenas: > - Firebase Services (Auth, Firestore, Storage) - BaaS > - Ollama LLM auto-hospedado > - Lógica de negócio implementada em Dart no Flutter --- ## 🏗️ WEEK 1-2: FIREBASE FOUNDATION ### Task 1.1: Firebase Project Setup **Status**: ✅ COMPLETED #### Subtasks: - ✅ Create Firebase project in Google Cloud Console - ✅ Enable required Firebase services: - ✅ Firebase Authentication - ✅ Cloud Firestore - ✅ Cloud Storage - ❌ Cloud Functions - NOT IMPLEMENTED (not needed) - ✅ Firebase Analytics - ✅ Configure project settings - ✅ Enable Ollama API access (self-hosted) #### Actual Configuration: **pubspec.yaml dependencies:** ```yaml dependencies: firebase_core: ^2.25.4 firebase_auth: ^4.17.8 cloud_firestore: ^4.15.8 firebase_storage: ^11.6.9 firebase_analytics: ^10.8.0 firebase_crashlytics: ^3.5.7 ``` **Ollama Configuration (in rag_ai_service.dart):** ```dart static const String _baseUrl = 'http://89.114.196.110:11434/api/chat'; static const String _model = 'qwen3-coder:30b'; ``` --- ### Task 1.2: Firestore Database Schema **Priority**: Critical **Estimated Time**: 8 hours **Dependencies**: Task 1.1 #### Subtasks: - [ ] Design collection structure - [ ] Create security rules - [ ] Set up indexes - [ ] Initialize sample data - [ ] Configure data validation #### Collection Schema: **schools/{schoolId}** ```typescript interface School { name: string; email: string; address: string; phone: string; settings: { curriculum: string[]; language: string; timezone: string; policies: { allowExternalKnowledge: boolean; fallbackMode: string; minRetrievalConfidence: number; }; }; subscription: { plan: 'free' | 'premium' | 'enterprise'; maxStudents: number; maxTeachers: number; expiresAt: Timestamp; }; createdAt: Timestamp; updatedAt: Timestamp; isActive: boolean; } ``` **users/{userId}** ```typescript interface User { schoolId: string; role: 'student' | 'teacher' | 'admin'; email: string; profile: { name: string; avatar?: string; gradeLevel?: number; // for students subjects?: string[]; // for teachers bio?: string; }; preferences: { language: string; notifications: boolean; darkMode: boolean; learningStyle?: 'visual' | 'auditory' | 'kinesthetic'; }; lastLogin: Timestamp; createdAt: Timestamp; updatedAt: Timestamp; isActive: boolean; emailVerified: boolean; } ``` **learningStates/{studentId}** ```typescript interface LearningState { studentId: string; schoolId: string; profile: { name: string; gradeLevel: number; subjects: string[]; learningStylePreference?: string; }; conceptStates: { [conceptId: string]: { name: string; mastery: number; // 0-1 confidence: number; // 0-1 engagement: { timesReviewed: number; totalTimeMinutes: number; lastActivity: Timestamp; daysSinceReview: number; }; misconceptions: { id: string; description: string; severity: 'low' | 'medium' | 'high'; firstDetected: Timestamp; lastAddressed: Timestamp; resolved: boolean; }[]; performance: { quizAttempts: number; quizScores: number[]; averageQuizScore: number; problemAccuracy: number; responseTimeAvgSeconds: number; }; forgettingCurve: { decayRate: number; estimatedRetention: number; nextReviewDate: Timestamp; }; }; }; spacedRepetition: { nextReviewDue: { conceptId: string; dueDate: Timestamp; priority: 'low' | 'medium' | 'high'; }[]; algorithm: 'sm2'; // Super Memo 2 }; learningGoals: { goalId: string; conceptId: string; targetMastery: number; deadline: Timestamp; progress: number; createdAt: Timestamp; }[]; adaptiveDifficulty: { currentLevel: number; // 1-6 Bloom's comfortableMin: number; comfortableMax: number; lastAdjusted: Timestamp; }; preferences: { modePreference: string; exampleFrequency: string; hintStyle: string; feedbackFrequency: string; }; metadata: { updatedAt: Timestamp; lastQuizDate: Timestamp; totalInteractions: number; dailyActiveDays: number; }; } ``` **contentChunks/{chunkId}** ```typescript interface ContentChunk { id: string; text: string; concept: string; subConcept?: string; subject: string; unit: string; pedagogy: { bloomLevel: number; // 1-6 difficulty: number; // 0-1 estimatedLearningTimeMinutes: number; abstractLevel: 'low' | 'medium' | 'high'; }; prerequisites: { conceptId: string; name: string; requiredMastery: number; }[]; content: { type: 'explanation' | 'example' | 'exercise' | 'assessment'; explanationChunks?: string[]; examples?: string[]; exercises?: string[]; quizQuestions?: string[]; }; commonMisconceptions: { id: string; description: string; remedialContent: string[]; frequency: number; }[]; relatedConcepts: string[]; realWorldApplications: string[]; embeddingVectorId: string; source: { documentId: string; fileName: string; page?: number; section?: string; teacherId: string; }; metadata: { createdAt: Timestamp; lastUpdated: Timestamp; author: string; version: string; qualityScore: number; isFlagged: boolean; flagReason?: string; }; tokens: number; language: string; } ``` **quizzes/{quizId}** ```typescript interface Quiz { id: string; teacherId: string; schoolId: string; title: string; description: string; subject: string; concept: string; difficulty: number; bloomLevel: number; settings: { timeLimitMinutes: number; allowReview: boolean; showResults: boolean; randomizeQuestions: boolean; randomizeOptions: boolean; }; questions: { id: string; type: 'multiple_choice' | 'short_answer' | 'true_false' | 'essay'; text: string; options?: string[]; // for multiple choice correctAnswer: string; explanation?: string; points: number; difficulty: number; conceptId: string; prerequisites?: string[]; }[]; metadata: { createdAt: Timestamp; lastUpdated: Timestamp; version: number; isActive: boolean; totalAttempts: number; averageScore: number; averageTimeMinutes: number; }; } ``` **quizAttempts/{attemptId}** ```typescript interface QuizAttempt { id: string; quizId: string; studentId: string; schoolId: string; answers: { [questionId: string]: { answer: string; isCorrect: boolean; timeSpentSeconds: number; attempts: number; }; }; score: number; maxScore: number; percentage: number; startedAt: Timestamp; completedAt: Timestamp; durationSeconds: number; feedback: { overall: string; strengths: string[]; improvements: string[]; nextSteps: string[]; }; misconceptionsIdentified: string[]; metadata: { device: string; browser: string; ipAddress: string; }; } ``` **interactions/{interactionId}** ```typescript interface Interaction { id: string; studentId: string; schoolId: string; type: 'question' | 'quiz' | 'feedback' | 'exploration'; query: string; response: string; mode: 'EXPLANATION' | 'TUTOR' | 'EXAM' | 'QUIZ' | 'EXPLORATION' | 'REMEDIAL'; retrievedChunks: { chunkId: string; confidence: number; relevanceScore: number; }[]; llmMetadata: { promptTokens: number; completionTokens: number; totalTokens: number; latencyMs: number; model: string; temperature: number; }; hallucinationScore: number; retrievalHitRate: number; contextOverlap: number; feedback?: { comprehension: 'understood' | 'partial' | 'confused'; difficulty: 'easy' | 'appropriate' | 'hard'; clarity: 'confusing' | 'ok' | 'clear'; comment?: string; timestamp: Timestamp; }; createdAt: Timestamp; metadata: { device: string; sessionId: string; ipAddress: string; }; } ``` **auditLogs/{logId}** ```typescript interface AuditLog { id: string; userId: string; schoolId: string; action: string; resource: string; resourceId: string; details: { oldValue?: any; newValue?: any; changes?: string[]; }; timestamp: Timestamp; ipAddress: string; userAgent: string; status: 'success' | 'failed'; errorMessage?: string; } ``` #### Security Rules: **firestore.rules** ```javascript rules_version = '2'; service cloud.firestore { match /databases/{database}/documents { // Helper functions function isAuthenticated() { return request.auth != null; } function isSameSchool(schoolId) { return isAuthenticated() && get(/databases/$(database)/documents/users/$(request.auth.uid)).data.schoolId == schoolId; } function hasRole(role) { return isAuthenticated() && get(/databases/$(database)/documents/users/$(request.auth.uid)).data.role == role; } // Schools collection - admin only match /schools/{schoolId} { allow read, write: if hasRole('admin'); } // Users collection match /users/{userId} { allow read: if isAuthenticated() && (request.auth.uid == userId || isSameSchool(resource.data.schoolId)); allow write: if isAuthenticated() && (request.auth.uid == userId || hasRole('admin')); allow create: if isAuthenticated() && request.auth.uid == userId && request.resource.data.role in ['student', 'teacher', 'admin']; } // Learning states - students can only access their own match /learningStates/{studentId} { allow read: if isAuthenticated() && (request.auth.uid == studentId || isSameSchool(resource.data.schoolId)); allow write: if isAuthenticated() && (request.auth.uid == studentId || hasRole('teacher') || hasRole('admin')); } // Content chunks - authenticated users from same school match /contentChunks/{chunkId} { allow read: if isAuthenticated() && isSameSchool(resource.data.schoolId); allow write: if isAuthenticated() && (hasRole('teacher') || hasRole('admin')) && isSameSchool(resource.data.schoolId); } // Quizzes match /quizzes/{quizId} { allow read: if isAuthenticated() && isSameSchool(resource.data.schoolId); allow write: if isAuthenticated() && (request.auth.uid == resource.data.teacherId || hasRole('admin')); allow create: if isAuthenticated() && hasRole('teacher') && request.resource.data.teacherId == request.auth.uid; } // Quiz attempts match /quizAttempts/{attemptId} { allow read: if isAuthenticated() && (request.auth.uid == resource.data.studentId || isSameSchool(resource.data.schoolId)); allow write: if isAuthenticated() && (request.auth.uid == resource.data.studentId || hasRole('teacher')); allow create: if isAuthenticated() && request.auth.uid == request.resource.data.studentId; } // Interactions match /interactions/{interactionId} { allow read: if isAuthenticated() && (request.auth.uid == resource.data.studentId || hasRole('teacher')); allow write: if isAuthenticated() && (request.auth.uid == resource.data.studentId || hasRole('teacher')); allow create: if isAuthenticated() && request.auth.uid == request.resource.data.studentId; } // Audit logs - read only for admins match /auditLogs/{logId} { allow read: if hasRole('admin'); allow write: if hasRole('admin'); } } } ``` #### Indexes: **firestore.indexes.json** ```json { "indexes": [ { "collectionGroup": "contentChunks", "queryScope": "COLLECTION", "fields": [ { "fieldPath": "schoolId", "order": "ASCENDING" }, { "fieldPath": "concept", "order": "ASCENDING" }, { "fieldPath": "pedagogy.difficulty", "order": "ASCENDING" } ] }, { "collectionGroup": "contentChunks", "queryScope": "COLLECTION", "fields": [ { "fieldPath": "schoolId", "order": "ASCENDING" }, { "fieldPath": "subject", "order": "ASCENDING" }, { "fieldPath": "unit", "order": "ASCENDING" } ] }, { "collectionGroup": "interactions", "queryScope": "COLLECTION", "fields": [ { "fieldPath": "studentId", "order": "ASCENDING" }, { "fieldPath": "createdAt", "order": "DESCENDING" } ] }, { "collectionGroup": "quizAttempts", "queryScope": "COLLECTION", "fields": [ { "fieldPath": "studentId", "order": "ASCENDING" }, { "fieldPath": "completedAt", "order": "DESCENDING" } ] }, { "collectionGroup": "quizAttempts", "queryScope": "COLLECTION", "fields": [ { "fieldPath": "quizId", "order": "ASCENDING" }, { "fieldPath": "score", "order": "DESCENDING" } ] } ], "fieldOverrides": [] } ``` --- ### Task 1.3: Cloud Functions Setup **Priority**: Critical **Estimated Time**: 6 hours **Dependencies**: Task 1.2 #### Subtasks: - [ ] Initialize Cloud Functions project - [ ] Set up TypeScript configuration - [ ] Configure environment variables - [ ] Set up deployment scripts - [ ] Create function templates #### Project Structure: ``` functions/ ├── src/ │ ├── index.ts │ ├── config/ │ │ ├── firebase.ts │ │ ├── llm.ts │ │ └── database.ts │ ├── services/ │ │ ├── auth.service.ts │ │ ├── rag.service.ts │ │ ├── llm.service.ts │ │ ├── analytics.service.ts │ │ └── storage.service.ts │ ├── middleware/ │ │ ├── auth.middleware.ts │ │ ├── validation.middleware.ts │ │ ├── rate-limit.middleware.ts │ │ └── error.middleware.ts │ ├── models/ │ │ ├── user.model.ts │ │ ├── content.model.ts │ │ ├── quiz.model.ts │ │ └── interaction.model.ts │ ├── utils/ │ │ ├── logger.ts │ │ ├── validators.ts │ │ ├── helpers.ts │ │ └── constants.ts │ └── functions/ │ ├── auth/ │ │ ├── signUp.ts │ │ ├── signIn.ts │ │ └── resetPassword.ts │ ├── content/ │ │ ├── uploadContent.ts │ │ ├── processContent.ts │ │ └── searchContent.ts │ ├── tutor/ │ │ ├── askTutor.ts │ │ ├── submitFeedback.ts │ │ └── getRecommendations.ts │ ├── quiz/ │ │ ├── createQuiz.ts │ │ ├── submitQuiz.ts │ │ └── getResults.ts │ └── analytics/ │ ├── getStudentProgress.ts │ ├── getClassAnalytics.ts │ └── getSystemMetrics.ts ├── package.json ├── tsconfig.json ├── .eslintrc.js └── .env.example ``` #### Configuration Files: **package.json** ```json { "name": "teachit-functions", "description": "Cloud Functions for AI Study Assistant", "scripts": { "build": "tsc", "build:watch": "tsc --watch", "serve": "npm run build && firebase emulators:start --only functions", "shell": "npm run build && firebase functions:shell", "start": "npm run shell", "deploy": "firebase deploy --only functions", "logs": "firebase functions:log" }, "engines": { "node": "18" }, "main": "lib/index.js", "dependencies": { "@google-cloud/firestore": "^6.7.0", "@google-cloud/storage": "^6.11.0", "firebase-admin": "^11.10.1", "firebase-functions": "^4.4.1", "openai": "^4.20.1", "anthropic": "^0.6.3", "sentence-transformers": "^0.0.1", "faiss-node": "^0.5.1", "pdf-parse": "^1.1.1", "mammoth": "^1.6.0", "express": "^4.18.2", "cors": "^2.8.5", "helmet": "^7.0.0", "express-rate-limit": "^6.10.0", "joi": "^17.9.2", "winston": "^3.10.0", "dotenv": "^16.3.1", "uuid": "^9.0.0", "bcryptjs": "^2.4.3", "jsonwebtoken": "^9.0.2" }, "devDependencies": { "@types/express": "^4.17.17", "@types/cors": "^2.8.13", "@types/uuid": "^9.0.2", "@types/bcryptjs": "^2.4.2", "@types/jsonwebtoken": "^9.0.2", "typescript": "^5.1.6", "@typescript-eslint/eslint-plugin": "^6.2.1", "@typescript-eslint/parser": "^6.2.1", "eslint": "^8.46.0", "eslint-config-google": "^0.14.0", "eslint-plugin-import": "^2.28.0", "firebase-functions-test": "^3.1.0" }, "private": true } ``` **tsconfig.json** ```json { "compilerOptions": { "module": "commonjs", "noImplicitReturns": true, "noUnusedLocals": true, "outDir": "lib", "sourceMap": true, "strict": true, "target": "es2017", "resolveJsonModule": true, "esModuleInterop": true, "skipLibCheck": true, "forceConsistentCasingInFileNames": true }, "compileOnSave": true, "include": [ "src" ], "exclude": [ "node_modules", "lib" ] } ``` --- ## 🔐 WEEK 3-4: AUTHENTICATION & USER MANAGEMENT ### Task 2.1: Authentication Functions **Priority**: Critical **Estimated Time**: 12 hours **Dependencies**: Task 1.3 #### Subtasks: - [ ] Implement user registration - [ ] Implement user login - [ ] Implement password reset - [ ] Add email verification - [ ] Create user profile management - [ ] Implement role-based access control #### Implementation: **src/functions/auth/signUp.ts** ```typescript import { https, CallableContext } from 'firebase-functions/v1'; import { getFirestore } from 'firebase-admin/firestore'; import { getAuth } from 'firebase-admin/auth'; import { CreateUserRequest, UserResponse } from '../../models/user.model'; import { validateCreateUser } from '../../utils/validators'; import { logger } from '../../utils/logger'; export const signUp = https.onCall(async (data: CreateUserRequest, context: CallableContext) => { try { // Validate input const validation = validateCreateUser(data); if (!validation.isValid) { throw new https.HttpsError('invalid-argument', validation.errors.join(', ')); } // Check if user already exists const auth = getAuth(); const existingUser = await auth.getUserByEmail(data.email); if (existingUser) { throw new https.HttpsError('already-exists', 'User with this email already exists'); } // Get school and validate const db = getFirestore(); const schoolDoc = await db.collection('schools').doc(data.schoolId).get(); if (!schoolDoc.exists) { throw new https.HttpsError('not-found', 'School not found'); } const school = schoolDoc.data(); if (!school.isActive) { throw new https.HttpsError('permission-denied', 'School is not active'); } // Check subscription limits const usersSnapshot = await db.collection('users') .where('schoolId', '==', data.schoolId) .where('isActive', '==', true) .get(); const studentCount = usersSnapshot.docs.filter(doc => doc.data().role === 'student').length; const teacherCount = usersSnapshot.docs.filter(doc => doc.data().role === 'teacher').length; if (data.role === 'student' && studentCount >= school.subscription.maxStudents) { throw new https.HttpsError('resource-exhausted', 'School has reached maximum student limit'); } if (data.role === 'teacher' && teacherCount >= school.subscription.maxTeachers) { throw new https.HttpsError('resource-exhausted', 'School has reached maximum teacher limit'); } // Create Firebase Auth user const userRecord = await auth.createUser({ email: data.email, password: data.password, displayName: data.profile.name, emailVerified: false, }); // Create user document const userDoc = { uid: userRecord.uid, schoolId: data.schoolId, role: data.role, email: data.email, profile: data.profile, preferences: { language: school.settings.language || 'en', notifications: true, darkMode: false, learningStyle: data.profile.learningStyle, }, createdAt: new Date(), updatedAt: new Date(), lastLogin: new Date(), isActive: true, emailVerified: false, }; await db.collection('users').doc(userRecord.uid).set(userDoc); // Create learning state for students if (data.role === 'student') { const learningState = { studentId: userRecord.uid, schoolId: data.schoolId, profile: { name: data.profile.name, gradeLevel: data.profile.gradeLevel || 10, subjects: [], }, conceptStates: {}, spacedRepetition: { nextReviewDue: [], algorithm: 'sm2', }, learningGoals: [], adaptiveDifficulty: { currentLevel: 2, comfortableMin: 1.5, comfortableMax: 2.8, lastAdjusted: new Date(), }, preferences: { modePreference: 'EXPLANATION', exampleFrequency: 'high', hintStyle: 'guided_questions', feedbackFrequency: 'immediate', }, metadata: { updatedAt: new Date(), totalInteractions: 0, dailyActiveDays: 0, }, }; await db.collection('learningStates').doc(userRecord.uid).set(learningState); } // Send email verification await auth.generateEmailVerificationLink(data.email); // Log audit await db.collection('auditLogs').add({ userId: userRecord.uid, schoolId: data.schoolId, action: 'user_created', resource: 'users', resourceId: userRecord.uid, details: { role: data.role, email: data.email, }, timestamp: new Date(), ipAddress: context.rawRequest.ip, userAgent: context.rawRequest.headers['user-agent'], status: 'success', }); logger.info(`User created successfully: ${userRecord.uid}`); return { success: true, user: { uid: userRecord.uid, email: data.email, role: data.role, profile: data.profile, }, } as UserResponse; } catch (error) { logger.error('Error creating user:', error); if (error instanceof https.HttpsError) { throw error; } throw new https.HttpsError('internal', 'Failed to create user'); } }); ``` **src/functions/auth/signIn.ts** ```typescript import { https, CallableContext } from 'firebase-functions/v1'; import { getFirestore } from 'firebase-admin/firestore'; import { getAuth } from 'firebase-admin/auth'; import { SignInRequest, UserResponse } from '../../models/user.model'; import { validateSignIn } from '../../utils/validators'; import { logger } from '../../utils/logger'; export const signIn = https.onCall(async (data: SignInRequest, context: CallableContext) => { try { // Validate input const validation = validateSignIn(data); if (!validation.isValid) { throw new https.HttpsError('invalid-argument', validation.errors.join(', ')); } // Authenticate user const auth = getAuth(); let userRecord; try { userRecord = await auth.getUserByEmail(data.email); } catch (error) { throw new https.HttpsError('not-found', 'User not found'); } // Check if user is active const db = getFirestore(); const userDoc = await db.collection('users').doc(userRecord.uid).get(); if (!userDoc.exists) { throw new https.HttpsError('not-found', 'User profile not found'); } const user = userDoc.data(); if (!user.isActive) { throw new https.HttpsError('permission-denied', 'Account is deactivated'); } // Update last login await userDoc.ref.update({ lastLogin: new Date(), updatedAt: new Date(), }); // Log audit await db.collection('auditLogs').add({ userId: userRecord.uid, schoolId: user.schoolId, action: 'user_sign_in', resource: 'users', resourceId: userRecord.uid, timestamp: new Date(), ipAddress: context.rawRequest.ip, userAgent: context.rawRequest.headers['user-agent'], status: 'success', }); logger.info(`User signed in successfully: ${userRecord.uid}`); return { success: true, user: { uid: userRecord.uid, email: user.email, role: user.role, profile: user.profile, preferences: user.preferences, }, } as UserResponse; } catch (error) { logger.error('Error signing in user:', error); if (error instanceof https.HttpsError) { throw error; } throw new https.HttpsError('internal', 'Failed to sign in user'); } }); ``` --- ### Task 2.2: User Profile Management **Priority**: High **Estimated Time**: 8 hours **Dependencies**: Task 2.1 #### Subtasks: - [ ] Implement profile update function - [ ] Add avatar upload functionality - [ ] Create preferences management - [ ] Implement password change - [ ] Add account deletion --- ## 📚 WEEK 5-6: CONTENT MANAGEMENT ### Task 3.1: Content Upload & Processing **Priority**: High **Estimated Time**: 16 hours **Dependencies**: Task 2.2 #### Subtasks: - [ ] Implement file upload endpoint - [ ] Create PDF parsing service - [ ] Build content chunking algorithm - [ ] Add content quality validation - [ ] Implement metadata extraction - [ ] Create content indexing #### Implementation: **src/services/content.service.ts** ```typescript import { getFirestore } from 'firebase-admin/firestore'; import { getStorage } from 'firebase-admin/storage'; import * as pdfParse from 'pdf-parse'; import * as mammoth from 'mammoth'; import { ContentChunk, ProcessedContent } from '../models/content.model'; import { chunkContent, validateChunk } from '../utils/content-processor'; import { generateEmbedding } from './llm.service'; import { logger } from '../utils/logger'; export class ContentService { private db = getFirestore(); private storage = getStorage(); async processUploadedFile( teacherId: string, schoolId: string, file: Buffer, fileName: string, mimeType: string, metadata: any ): Promise { try { logger.info(`Processing file: ${fileName} for teacher: ${teacherId}`); // Extract text based on file type let text: string; let extractedMetadata: any = {}; switch (mimeType) { case 'application/pdf': const pdfData = await pdfParse(file); text = pdfData.text; extractedMetadata = { pageCount: pdfData.numpages, info: pdfData.info, }; break; case 'application/vnd.openxmlformats-officedocument.wordprocessingml.document': const docxResult = await mammoth.extractRawText({ buffer: file }); text = docxResult.value; extractedMetadata = { wordCount: text.split(/\s+/).length, }; break; case 'text/plain': text = file.toString('utf-8'); extractedMetadata = { wordCount: text.split(/\s+/).length, }; break; default: throw new Error(`Unsupported file type: ${mimeType}`); } // Validate extracted content if (!text || text.trim().length < 100) { throw new Error('Insufficient content extracted from file'); } // Process and chunk content const chunks = await this.chunkAndProcessContent( text, teacherId, schoolId, fileName, metadata, extractedMetadata ); // Generate embeddings for all chunks const chunksWithEmbeddings = await Promise.all( chunks.map(async (chunk) => { const embedding = await generateEmbedding(chunk.text); return { ...chunk, embeddingVectorId: `vec_${chunk.id}`, embedding, }; }) ); // Save chunks to Firestore const batch = this.db.batch(); chunksWithEmbeddings.forEach((chunk) => { const docRef = this.db.collection('contentChunks').doc(chunk.id); batch.set(docRef, chunk); }); await batch.commit(); // Update teacher's content count await this.db.collection('users').doc(teacherId).update({ 'metadata.contentCount': admin.firestore.FieldValue.increment(1), updatedAt: new Date(), }); // Log audit await this.db.collection('auditLogs').add({ userId: teacherId, schoolId, action: 'content_uploaded', resource: 'contentChunks', details: { fileName, chunkCount: chunks.length, mimeType, }, timestamp: new Date(), status: 'success', }); logger.info(`Successfully processed ${chunks.length} chunks from ${fileName}`); return { success: true, chunkCount: chunks.length, chunks: chunksWithEmbeddings, metadata: extractedMetadata, }; } catch (error) { logger.error(`Error processing file ${fileName}:`, error); throw error; } } private async chunkAndProcessContent( text: string, teacherId: string, schoolId: string, fileName: string, metadata: any, extractedMetadata: any ): Promise { const chunks: ContentChunk[] = []; // Detect sections (teacher markers or automatic) const sections = this.detectSections(text); for (const section of sections) { // Split section into smaller chunks const textChunks = this.splitIntoChunks(section.content, 400, 50); for (let i = 0; i < textChunks.length; i++) { const chunkText = textChunks[i]; const chunkId = `chunk_${Date.now()}_${i}`; // Extract pedagogical metadata const pedagogy = this.extractPedagogicalMetadata(chunkText, metadata); // Create chunk object const chunk: ContentChunk = { id: chunkId, text: chunkText, concept: section.concept || 'General', subConcept: section.subConcept, subject: metadata.subject || 'General', unit: metadata.unit || 'General', pedagogy: { bloomLevel: pedagogy.bloomLevel || 2, difficulty: pedagogy.difficulty || 0.5, estimatedLearningTimeMinutes: pedagogy.estimatedTime || 15, abstractLevel: pedagogy.abstractLevel || 'medium', }, prerequisites: pedagogy.prerequisites || [], content: { type: this.detectContentType(chunkText), }, commonMisconceptions: [], relatedConcepts: [], realWorldApplications: [], source: { documentId: `doc_${Date.now()}`, fileName, page: extractedMetadata.page, section: section.title, teacherId, }, metadata: { createdAt: new Date(), lastUpdated: new Date(), author: teacherId, version: '1.0', qualityScore: this.calculateQualityScore(chunkText), isFlagged: false, }, tokens: this.countTokens(chunkText), language: metadata.language || 'en', }; // Validate chunk if (validateChunk(chunk)) { chunks.push(chunk); } else { logger.warn(`Chunk ${chunkId} failed validation`); } } } return chunks; } private detectSections(text: string): Array<{ title: string; concept?: string; subConcept?: string; content: string; }> { const sections: Array<{ title: string; concept?: string; subConcept?: string; content: string; }> = []; // Look for teacher-defined markers const conceptStartRegex = /\[CONCEPT_START:\s*([^\]]+)\]/g; const conceptEndRegex = /\[CONCEPT_END\]/g; const exampleStartRegex = /\[EXAMPLE_START\]/g; const exampleEndRegex = /\[EXAMPLE_END\]/g; let currentSection: any = null; const lines = text.split('\n'); for (const line of lines) { const conceptMatch = conceptStartRegex.exec(line); if (conceptMatch) { // Save previous section if exists if (currentSection) { sections.push(currentSection); } // Start new section currentSection = { title: conceptMatch[1], concept: conceptMatch[1], content: '', }; continue; } if (conceptEndRegex.test(line)) { if (currentSection) { sections.push(currentSection); currentSection = null; } continue; } if (currentSection) { currentSection.content += line + '\n'; } } // Add last section if exists if (currentSection) { sections.push(currentSection); } // If no sections found, treat entire text as one section if (sections.length === 0) { sections.push({ title: 'Content', content: text, }); } return sections; } private splitIntoChunks(text: string, maxTokens: number, overlapTokens: number): string[] { const words = text.split(/\s+/); const chunks: string[] = []; let currentChunk: string[] = []; let currentTokens = 0; for (let i = 0; i < words.length; i++) { const word = words[i]; currentChunk.push(word); currentTokens++; // Check if we've reached max tokens if (currentTokens >= maxTokens) { chunks.push(currentChunk.join(' ')); // Start next chunk with overlap currentChunk = []; currentTokens = 0; // Add overlap words const overlapStart = Math.max(0, i - overlapTokens + 1); for (let j = overlapStart; j <= i; j++) { currentChunk.push(words[j]); currentTokens++; } } } // Add remaining words if (currentChunk.length > 0) { chunks.push(currentChunk.join(' ')); } return chunks; } private extractPedagogicalMetadata(text: string, metadata: any): any { // This would use NLP or rule-based extraction // For MVP, we'll use simple heuristics const pedagogy = { bloomLevel: 2, // Default to Understanding difficulty: 0.5, // Medium difficulty estimatedTime: 15, // 15 minutes abstractLevel: 'medium' as const, prerequisites: [], }; // Detect Bloom's level from text if (text.includes('define') || text.includes('identify')) { pedagogy.bloomLevel = 1; // Remember } else if (text.includes('analyze') || text.includes('compare')) { pedagogy.bloomLevel = 4; // Analyze } else if (text.includes('create') || text.includes('design')) { pedagogy.bloomLevel = 6; // Create } // Detect difficulty from complexity const sentences = text.split(/[.!?]+/).length; const avgWordsPerSentence = text.split(/\s+/).length / sentences; if (avgWordsPerSentence > 20) { pedagogy.difficulty = 0.8; // High difficulty } else if (avgWordsPerSentence < 10) { pedagogy.difficulty = 0.2; // Low difficulty } return pedagogy; } private detectContentType(text: string): 'explanation' | 'example' | 'exercise' | 'assessment' { const lowerText = text.toLowerCase(); if (lowerText.includes('example') || lowerText.includes('for example')) { return 'example'; } else if (lowerText.includes('exercise') || lowerText.includes('practice')) { return 'exercise'; } else if (lowerText.includes('question') || lowerText.includes('quiz')) { return 'assessment'; } else { return 'explanation'; } } private calculateQualityScore(text: string): number { let score = 0.5; // Base score // Length check if (text.length > 100 && text.length < 1000) { score += 0.2; } // Grammar check (simplified) const sentences = text.split(/[.!?]+/); if (sentences.length > 1) { score += 0.1; } // Example check if (text.toLowerCase().includes('example')) { score += 0.1; } // Definition check if (text.toLowerCase().includes('define') || text.toLowerCase().includes('is')) { score += 0.1; } return Math.min(1.0, score); } private countTokens(text: string): number { // Simple token estimation (roughly 4 characters per token) return Math.ceil(text.length / 4); } } ``` --- ### Task 3.2: Content Search & Retrieval **Priority**: High **Estimated Time**: 12 hours **Dependencies**: Task 3.1 #### Subtasks: - [ ] Implement keyword search (BM25) - [ ] Create vector similarity search - [ ] Build hybrid retrieval engine - [ ] Add metadata filtering - [ ] Implement ranking algorithm - [ ] Create content recommendations --- ## 🤖 WEEK 7-8: AI TUTOR INTEGRATION ### Task 4.1: RAG Pipeline Implementation **Priority**: High **Estimated Time**: 20 hours **Dependencies**: Task 3.2 #### Subtasks: - [ ] Implement query understanding - [ ] Create context retrieval - [ ] Build prompt assembly - [ ] Integrate LLM API - [ ] Add hallucination detection - [ ] Implement response filtering #### Implementation: **src/services/rag.service.ts** ```typescript import { getFirestore } from 'firebase-admin/firestore'; import { ContentChunk, RetrievalResult, RAGContext } from '../models/content.model'; import { ContentService } from './content.service'; import { LLMService } from './llm.service'; import { logger } from '../utils/logger'; export class RAGService { private db = getFirestore(); private contentService = new ContentService(); private llmService = new LLMService(); async processStudentQuery( studentId: string, query: string, mode: string = 'EXPLANATION' ): Promise { try { logger.info(`Processing query for student ${studentId}: "${query}"`); // Get student learning state const learningStateDoc = await this.db .collection('learningStates') .doc(studentId) .get(); if (!learningStateDoc.exists) { throw new Error('Student learning state not found'); } const learningState = learningStateDoc.data(); // Step 1: Query Understanding const queryAnalysis = await this.analyzeQuery(query, learningState); // Step 2: Context Retrieval const retrievalResult = await this.retrieveContext( query, queryAnalysis, learningState ); // Step 3: Check if we have sufficient context if (retrievalResult.chunks.length === 0) { return this.handleNoContext(query, queryAnalysis); } // Step 4: Build RAG Context const ragContext = this.buildRAGContext( query, queryAnalysis, retrievalResult, learningState, mode ); // Step 5: Generate Response const response = await this.llmService.generateResponse(ragContext); // Step 6: Post-processing const processedResponse = await this.postProcessResponse( response, retrievalResult, ragContext ); // Step 7: Log Interaction await this.logInteraction( studentId, query, processedResponse, retrievalResult, ragContext ); return processedResponse; } catch (error) { logger.error('Error processing student query:', error); throw error; } } private async analyzeQuery(query: string, learningState: any): Promise { const analysis = { intent: this.detectIntent(query), studentLevel: learningState.adaptiveDifficulty.currentLevel, detectedSubject: this.detectSubject(query), detectedConcept: this.detectConcept(query), queryComplexity: this.assessComplexity(query), expandedQuery: this.expandQuery(query), }; logger.info('Query analysis:', analysis); return analysis; } private detectIntent(query: string): string { const lowerQuery = query.toLowerCase(); if (lowerQuery.includes('what is') || lowerQuery.includes('define')) { return 'ask_concept'; } else if (lowerQuery.includes('how to') || lowerQuery.includes('solve')) { return 'solve_problem'; } else if (lowerQuery.includes('why') || lowerQuery.includes('explain')) { return 'explain_why'; } else if (lowerQuery.includes('example') || lowerQuery.includes('show me')) { return 'request_example'; } else { return 'general_question'; } } private detectSubject(query: string): string { const subjects = ['math', 'calculus', 'algebra', 'geometry', 'physics', 'chemistry']; const lowerQuery = query.toLowerCase(); for (const subject of subjects) { if (lowerQuery.includes(subject)) { return subject; } } return 'general'; } private detectConcept(query: string): string { // This would use a more sophisticated approach in production // For MVP, we'll use simple keyword matching const concepts = [ 'derivative', 'integral', 'limit', 'function', 'equation', 'velocity', 'acceleration', 'force', 'energy' ]; const lowerQuery = query.toLowerCase(); for (const concept of concepts) { if (lowerQuery.includes(concept)) { return concept; } } return 'unknown'; } private assessComplexity(query: string): number { const words = query.split(/\s+/); const sentences = query.split(/[.!?]+/).length; const avgWordsPerSentence = words.length / sentences; let complexity = 0.3; // Base complexity if (avgWordsPerSentence > 15) complexity += 0.2; if (words.length > 20) complexity += 0.2; if (query.includes('why') || query.includes('explain')) complexity += 0.1; if (query.includes('compare') || query.includes('analyze')) complexity += 0.2; return Math.min(1.0, complexity); } private expandQuery(query: string): string[] { // Simple query expansion using synonyms const expansions: string[] = [query]; const synonyms: { [key: string]: string[] } = { 'derivative': ['rate of change', 'slope', 'differentiation'], 'integral': ['antiderivative', 'integration', 'area under curve'], 'function': ['mapping', 'relation', 'transformation'], }; const lowerQuery = query.toLowerCase(); for (const [term, syns] of Object.entries(synonyms)) { if (lowerQuery.includes(term)) { for (const synonym of syns) { expansions.push(query.replace(new RegExp(term, 'gi'), synonym)); } } } return expansions; } private async retrieveContext( query: string, queryAnalysis: any, learningState: any ): Promise { const retrievalStrategies = [ this.keywordSearch.bind(this), this.vectorSearch.bind(this), this.metadataSearch.bind(this), ]; const allResults: ContentChunk[] = []; const strategyResults: { [strategy: string]: ContentChunk[] } = {}; // Run all retrieval strategies for (const strategy of retrievalStrategies) { try { const results = await strategy(query, queryAnalysis, learningState); strategyResults[strategy.name] = results; allResults.push(...results); } catch (error) { logger.error(`Error in ${strategy.name}:`, error); } } // Deduplicate and rank results const uniqueResults = this.deduplicateResults(allResults); const rankedResults = this.rankResults(uniqueResults, query, queryAnalysis); // Filter by difficulty level const filteredResults = rankedResults.filter( chunk => chunk.pedagogy.difficulty <= learningState.adaptiveDifficulty.currentLevel / 6 ); // Take top results const topResults = filteredResults.slice(0, 5); return { chunks: topResults, totalFound: allResults.length, strategyResults, queryAnalysis, }; } private async keywordSearch( query: string, queryAnalysis: any, learningState: any ): Promise { const expandedQueries = queryAnalysis.expandedQuery; const results: ContentChunk[] = []; for (const expandedQuery of expandedQueries) { const snapshot = await this.db .collection('contentChunks') .where('schoolId', '==', learningState.schoolId) .where('text', 'array-contains', expandedQuery.split(/\s+/)[0]) .limit(10) .get(); snapshot.docs.forEach(doc => { const chunk = doc.data() as ContentChunk; chunk.id = doc.id; results.push(chunk); }); } return results; } private async vectorSearch( query: string, queryAnalysis: any, learningState: any ): Promise { // This would integrate with a vector database (FAISS, Weaviate, etc.) // For MVP, we'll use a simplified approach const queryEmbedding = await this.llmService.generateEmbedding(query); // In production, this would query the vector database // For now, return empty results return []; } private async metadataSearch( query: string, queryAnalysis: any, learningState: any ): Promise { let queryBuilder = this.db .collection('contentChunks') .where('schoolId', '==', learningState.schoolId); // Filter by subject if detected if (queryAnalysis.detectedSubject !== 'general') { queryBuilder = queryBuilder.where('subject', '==', queryAnalysis.detectedSubject); } // Filter by concept if detected if (queryAnalysis.detectedConcept !== 'unknown') { queryBuilder = queryBuilder.where('concept', '==', queryAnalysis.detectedConcept); } const snapshot = await queryBuilder.limit(10).get(); return snapshot.docs.map(doc => { const chunk = doc.data() as ContentChunk; chunk.id = doc.id; return chunk; }); } private deduplicateResults(results: ContentChunk[]): ContentChunk[] { const seen = new Set(); return results.filter(chunk => { const key = `${chunk.concept}_${chunk.text.substring(0, 50)}`; if (seen.has(key)) { return false; } seen.add(key); return true; }); } private rankResults( results: ContentChunk[], query: string, queryAnalysis: any ): ContentChunk[] { return results .map(chunk => ({ ...chunk, score: this.calculateRelevanceScore(chunk, query, queryAnalysis), })) .sort((a, b) => b.score - a.score); } private calculateRelevanceScore( chunk: ContentChunk, query: string, queryAnalysis: any ): number { let score = 0; // Text similarity (simplified) const queryWords = query.toLowerCase().split(/\s+/); const chunkWords = chunk.text.toLowerCase().split(/\s+/); const commonWords = queryWords.filter(word => chunkWords.includes(word)); score += (commonWords.length / queryWords.length) * 0.4; // Concept matching if (chunk.concept.toLowerCase().includes(queryAnalysis.detectedConcept.toLowerCase())) { score += 0.3; } // Subject matching if (chunk.subject.toLowerCase().includes(queryAnalysis.detectedSubject.toLowerCase())) { score += 0.2; } // Difficulty appropriateness const difficultyDiff = Math.abs(chunk.pedagogy.difficulty - queryAnalysis.studentLevel / 6); score += (1 - difficultyDiff) * 0.1; return score; } private buildRAGContext( query: string, queryAnalysis: any, retrievalResult: RetrievalResult, learningState: any, mode: string ): RAGContext { const contextText = retrievalResult.chunks .map(chunk => `[${chunk.concept}]\n${chunk.text}`) .join('\n\n'); return { query, mode, studentLevel: learningState.adaptiveDifficulty.currentLevel, contextText, retrievedChunks: retrievalResult.chunks, queryAnalysis, learningState, constraints: this.getModeConstraints(mode, learningState), }; } private getModeConstraints(mode: string, learningState: any): any { const baseConstraints = { maxTokens: 500, temperature: 0.7, includeExamples: true, avoidProofs: learningState.adaptiveDifficulty.currentLevel < 4, }; switch (mode) { case 'EXPLANATION': return { ...baseConstraints, bloomLevel: Math.min(learningState.adaptiveDifficulty.currentLevel, 3), style: 'explanatory', includeExamples: true, }; case 'TUTOR': return { ...baseConstraints, bloomLevel: learningState.adaptiveDifficulty.currentLevel, style: 'socratic', askQuestions: true, progressiveHints: true, }; case 'EXAM': return { ...baseConstraints, bloomLevel: 'any', style: 'formal', includeExamples: false, giveAnswers: false, }; case 'QUIZ': return { ...baseConstraints, bloomLevel: Math.min(learningState.adaptiveDifficulty.currentLevel, 2), style: 'interactive', immediateFeedback: true, }; default: return baseConstraints; } } private async handleNoContext(query: string, queryAnalysis: any): Promise { return { status: 'no_context', message: 'Sorry, I don\'t have content on that topic yet.', suggestions: await this.generateSuggestions(query, queryAnalysis), fallbackMode: 'partial_with_hint', }; } private async generateSuggestions(query: string, queryAnalysis: any): Promise { const suggestions: string[] = []; // Suggest learning prerequisites if (queryAnalysis.detectedConcept !== 'unknown') { suggestions.push(`Would you like to learn about prerequisites for ${queryAnalysis.detectedConcept}?`); } // Suggest related topics suggestions.push('Try asking about basic concepts first.'); suggestions.push('Would you like me to help you with a different topic?'); return suggestions; } private async postProcessResponse( response: any, retrievalResult: RetrievalResult, ragContext: RAGContext ): Promise { // Detect hallucination risk const hallucinationScore = this.detectHallucinationRisk( response.text, retrievalResult.chunks ); // Filter inappropriate content const filteredResponse = this.filterResponse(response.text, ragContext.constraints); return { ...response, text: filteredResponse, hallucinationScore, retrievalHitRate: retrievalResult.chunks.length > 0 ? 1 : 0, contextOverlap: this.calculateContextOverlap(response.text, retrievalResult.chunks), metadata: { chunksUsed: retrievalResult.chunks.length, mode: ragContext.mode, studentLevel: ragContext.studentLevel, }, }; } private detectHallucinationRisk(response: string, chunks: ContentChunk[]): number { // Simple hallucination detection const responseWords = response.toLowerCase().split(/\s+/); const contextWords = chunks .map(chunk => chunk.text.toLowerCase()) .join(' ') .split(/\s+/); const commonWords = responseWords.filter(word => contextWords.includes(word)); const overlap = commonWords.length / responseWords.length; return 1 - overlap; // Higher score = higher risk } private filterResponse(response: string, constraints: any): string { let filtered = response; // Remove content that violates constraints if (constraints.avoidProofs) { filtered = filtered.replace(/proof|prove|theorem/gi, '[proof omitted]'); } // Ensure response length if (constraints.maxTokens && filtered.length > constraints.maxTokens * 4) { filtered = filtered.substring(0, constraints.maxTokens * 4) + '...'; } return filtered; } private calculateContextOverlap(response: string, chunks: ContentChunk[]): number { const responseWords = response.toLowerCase().split(/\s+/); const contextWords = chunks .map(chunk => chunk.text.toLowerCase()) .join(' ') .split(/\s+/); const commonWords = responseWords.filter(word => contextWords.includes(word)); return commonWords.length / responseWords.length; } private async logInteraction( studentId: string, query: string, response: any, retrievalResult: RetrievalResult, ragContext: RAGContext ): Promise { await this.db.collection('interactions').add({ studentId, schoolId: ragContext.learningState.schoolId, type: 'question', query, response: response.text, mode: ragContext.mode, retrievedChunks: retrievalResult.chunks.map(chunk => ({ chunkId: chunk.id, confidence: chunk.score || 0.5, relevanceScore: chunk.score || 0.5, })), llmMetadata: { promptTokens: response.promptTokens || 0, completionTokens: response.completionTokens || 0, totalTokens: response.totalTokens || 0, latencyMs: response.latencyMs || 0, model: response.model || 'unknown', temperature: ragContext.constraints.temperature, }, hallucinationScore: response.hallucinationScore || 0, retrievalHitRate: retrievalResult.chunks.length > 0 ? 1 : 0, contextOverlap: response.contextOverlap || 0, createdAt: new Date(), }); } } ``` --- ### Task 4.2: LLM Integration **Priority**: High **Estimated Time**: 12 hours **Dependencies**: Task 4.1 #### Subtasks: - [ ] Set up OpenAI/Anthropic API - [ ] Implement prompt engineering - [ ] Create response generation - [ ] Add token counting - [ ] Implement rate limiting - [ ] Add error handling --- ## 📊 WEEK 9-10: QUIZ SYSTEM & ANALYTICS ### Task 5.1: Quiz Management **Priority**: High **Estimated Time**: 16 hours **Dependencies**: Task 4.2 #### Subtasks: - [ ] Create quiz creation endpoint - [ ] Implement quiz taking logic - [ ] Add automatic grading - [ ] Build quiz analytics - [ ] Create quiz history - [ ] Implement quiz recommendations --- ### Task 5.2: Analytics Engine **Priority**: Medium **Estimated Time**: 12 hours **Dependencies**: Task 5.1 #### Subtasks: - [ ] Implement student progress tracking - [ ] Create class analytics - [ ] Build learning recommendations - [ ] Add performance metrics - [ ] Create analytics dashboard data - [ ] Implement export functionality --- ## 🧪 WEEK 11-12: TESTING & DEPLOYMENT ### Task 6.1: Testing Suite **Priority**: High **Estimated Time**: 16 hours **Dependencies**: All previous tasks #### Subtasks: - [ ] Write unit tests for all services - [ ] Create integration tests - [ ] Test Firebase security rules - [ ] Load testing for APIs - [ ] Error scenario testing - [ ] Performance benchmarking --- ### Task 6.2: Production Deployment **Priority**: High **Estimated Time**: 8 hours **Dependencies**: Task 6.1 #### Subtasks: - [ ] Configure production environment - [ ] Set up monitoring and logging - [ ] Configure alerts - [ ] Deploy to production - [ ] Run smoke tests - [ ] Monitor performance --- ## 📋 DELIVERABLES ### Week 2 Deliverables - [ ] Firebase project with complete configuration - [ ] Firestore database with all collections - [ ] Security rules and indexes - [ ] Cloud Functions project structure ### Week 4 Deliverables - [ ] Complete authentication system - [ ] User management functions - [ ] Role-based access control - [ ] Profile management ### Week 6 Deliverables - [ ] Content upload and processing - [ ] Content search and retrieval - [ ] Content management system - [ ] Quality validation ### Week 8 Deliverables - [ ] Complete RAG pipeline - [ ] LLM integration - [ ] AI tutor functionality - [ ] Response generation and filtering ### Week 10 Deliverables - [ ] Quiz system - [ ] Analytics engine - [ ] Progress tracking - [ ] Recommendation system ### Week 12 Deliverables - [ ] Complete test suite - [ ] Production deployment - [ ] Monitoring and logging - [ ] Documentation --- ## 🔧 ENVIRONMENT CONFIGURATION ### Development Environment ```bash # Firebase emulators firebase emulators:start # Local functions npm run serve # Test data seeding npm run seed:data ``` ### Production Environment ```bash # Deploy all functions firebase deploy --only functions # Deploy specific function firebase deploy --only functions:askTutor # View logs firebase functions:log ``` --- ## 📈 MONITORING & LOGGING ### Key Metrics to Track - API response times - Error rates - LLM token usage - Retrieval hit rates - User engagement - System performance ### Alert Configuration - High error rates (>5%) - Slow responses (>3s) - LLM quota exhaustion - Database connection issues - Storage capacity warnings --- ## 🛡️ SECURITY CONSIDERATIONS ### API Security - Rate limiting per user - Input validation and sanitization - SQL injection prevention - XSS protection - CORS configuration ### Data Security - Encryption at rest and in transit - Access control validation - Audit logging - Data retention policies - GDPR compliance --- *Last Updated: 2026-05-06* *Version: 1.0.0* *Backend Lead: Cloud Functions Development Team*