Files
LearnIT/docs/BACKEND_MVP_TASKS.md
2026-05-06 20:16:11 +01:00

2232 lines
58 KiB
Markdown

# Backend MVP Tasks - AI Study Assistant
## 🔧 MVP BACKEND ROADMAP (8-12 WEEKS)
---
## 🏗️ WEEK 1-2: FIREBASE FOUNDATION
### Task 1.1: Firebase Project Setup
**Priority**: Critical
**Estimated Time**: 6 hours
**Dependencies**: None
#### Subtasks:
- [ ] Create Firebase project in Google Cloud Console
- [ ] Enable required Firebase services:
- [ ] Firebase Authentication
- [ ] Cloud Firestore
- [ ] Cloud Storage
- [ ] Cloud Functions
- [ ] Firebase Analytics
- [ ] Configure project settings
- [ ] Set up billing account (if needed)
- [ ] Enable API access for LLM services
#### Detailed Steps:
1. **Create Firebase Project**
```bash
# Using Firebase CLI
firebase projects create teachit-ai-assistant
firebase use teachit-ai-assistant
```
2. **Enable Services**
```bash
# Enable Authentication
firebase auth --enable
# Enable Firestore
firebase firestore:databases:create
# Enable Storage
firebase storage:buckets:create teachit-content
# Enable Functions
firebase functions:config:set
```
3. **Project Configuration**
```json
// firebase.json
{
"firestore": {
"rules": "firestore.rules",
"indexes": "firestore.indexes.json"
},
"storage": {
"rules": "storage.rules"
},
"functions": {
"predeploy": [
"npm --prefix \"$RESOURCE_DIR\" run lint",
"npm --prefix \"$RESOURCE_DIR\" run build"
],
"source": "functions"
}
}
```
---
### Task 1.2: Firestore Database Schema
**Priority**: Critical
**Estimated Time**: 8 hours
**Dependencies**: Task 1.1
#### Subtasks:
- [ ] Design collection structure
- [ ] Create security rules
- [ ] Set up indexes
- [ ] Initialize sample data
- [ ] Configure data validation
#### Collection Schema:
**schools/{schoolId}**
```typescript
interface School {
name: string;
email: string;
address: string;
phone: string;
settings: {
curriculum: string[];
language: string;
timezone: string;
policies: {
allowExternalKnowledge: boolean;
fallbackMode: string;
minRetrievalConfidence: number;
};
};
subscription: {
plan: 'free' | 'premium' | 'enterprise';
maxStudents: number;
maxTeachers: number;
expiresAt: Timestamp;
};
createdAt: Timestamp;
updatedAt: Timestamp;
isActive: boolean;
}
```
**users/{userId}**
```typescript
interface User {
schoolId: string;
role: 'student' | 'teacher' | 'admin';
email: string;
profile: {
name: string;
avatar?: string;
gradeLevel?: number; // for students
subjects?: string[]; // for teachers
bio?: string;
};
preferences: {
language: string;
notifications: boolean;
darkMode: boolean;
learningStyle?: 'visual' | 'auditory' | 'kinesthetic';
};
lastLogin: Timestamp;
createdAt: Timestamp;
updatedAt: Timestamp;
isActive: boolean;
emailVerified: boolean;
}
```
**learningStates/{studentId}**
```typescript
interface LearningState {
studentId: string;
schoolId: string;
profile: {
name: string;
gradeLevel: number;
subjects: string[];
learningStylePreference?: string;
};
conceptStates: {
[conceptId: string]: {
name: string;
mastery: number; // 0-1
confidence: number; // 0-1
engagement: {
timesReviewed: number;
totalTimeMinutes: number;
lastActivity: Timestamp;
daysSinceReview: number;
};
misconceptions: {
id: string;
description: string;
severity: 'low' | 'medium' | 'high';
firstDetected: Timestamp;
lastAddressed: Timestamp;
resolved: boolean;
}[];
performance: {
quizAttempts: number;
quizScores: number[];
averageQuizScore: number;
problemAccuracy: number;
responseTimeAvgSeconds: number;
};
forgettingCurve: {
decayRate: number;
estimatedRetention: number;
nextReviewDate: Timestamp;
};
};
};
spacedRepetition: {
nextReviewDue: {
conceptId: string;
dueDate: Timestamp;
priority: 'low' | 'medium' | 'high';
}[];
algorithm: 'sm2'; // Super Memo 2
};
learningGoals: {
goalId: string;
conceptId: string;
targetMastery: number;
deadline: Timestamp;
progress: number;
createdAt: Timestamp;
}[];
adaptiveDifficulty: {
currentLevel: number; // 1-6 Bloom's
comfortableMin: number;
comfortableMax: number;
lastAdjusted: Timestamp;
};
preferences: {
modePreference: string;
exampleFrequency: string;
hintStyle: string;
feedbackFrequency: string;
};
metadata: {
updatedAt: Timestamp;
lastQuizDate: Timestamp;
totalInteractions: number;
dailyActiveDays: number;
};
}
```
**contentChunks/{chunkId}**
```typescript
interface ContentChunk {
id: string;
text: string;
concept: string;
subConcept?: string;
subject: string;
unit: string;
pedagogy: {
bloomLevel: number; // 1-6
difficulty: number; // 0-1
estimatedLearningTimeMinutes: number;
abstractLevel: 'low' | 'medium' | 'high';
};
prerequisites: {
conceptId: string;
name: string;
requiredMastery: number;
}[];
content: {
type: 'explanation' | 'example' | 'exercise' | 'assessment';
explanationChunks?: string[];
examples?: string[];
exercises?: string[];
quizQuestions?: string[];
};
commonMisconceptions: {
id: string;
description: string;
remedialContent: string[];
frequency: number;
}[];
relatedConcepts: string[];
realWorldApplications: string[];
embeddingVectorId: string;
source: {
documentId: string;
fileName: string;
page?: number;
section?: string;
teacherId: string;
};
metadata: {
createdAt: Timestamp;
lastUpdated: Timestamp;
author: string;
version: string;
qualityScore: number;
isFlagged: boolean;
flagReason?: string;
};
tokens: number;
language: string;
}
```
**quizzes/{quizId}**
```typescript
interface Quiz {
id: string;
teacherId: string;
schoolId: string;
title: string;
description: string;
subject: string;
concept: string;
difficulty: number;
bloomLevel: number;
settings: {
timeLimitMinutes: number;
allowReview: boolean;
showResults: boolean;
randomizeQuestions: boolean;
randomizeOptions: boolean;
};
questions: {
id: string;
type: 'multiple_choice' | 'short_answer' | 'true_false' | 'essay';
text: string;
options?: string[]; // for multiple choice
correctAnswer: string;
explanation?: string;
points: number;
difficulty: number;
conceptId: string;
prerequisites?: string[];
}[];
metadata: {
createdAt: Timestamp;
lastUpdated: Timestamp;
version: number;
isActive: boolean;
totalAttempts: number;
averageScore: number;
averageTimeMinutes: number;
};
}
```
**quizAttempts/{attemptId}**
```typescript
interface QuizAttempt {
id: string;
quizId: string;
studentId: string;
schoolId: string;
answers: {
[questionId: string]: {
answer: string;
isCorrect: boolean;
timeSpentSeconds: number;
attempts: number;
};
};
score: number;
maxScore: number;
percentage: number;
startedAt: Timestamp;
completedAt: Timestamp;
durationSeconds: number;
feedback: {
overall: string;
strengths: string[];
improvements: string[];
nextSteps: string[];
};
misconceptionsIdentified: string[];
metadata: {
device: string;
browser: string;
ipAddress: string;
};
}
```
**interactions/{interactionId}**
```typescript
interface Interaction {
id: string;
studentId: string;
schoolId: string;
type: 'question' | 'quiz' | 'feedback' | 'exploration';
query: string;
response: string;
mode: 'EXPLANATION' | 'TUTOR' | 'EXAM' | 'QUIZ' | 'EXPLORATION' | 'REMEDIAL';
retrievedChunks: {
chunkId: string;
confidence: number;
relevanceScore: number;
}[];
llmMetadata: {
promptTokens: number;
completionTokens: number;
totalTokens: number;
latencyMs: number;
model: string;
temperature: number;
};
hallucinationScore: number;
retrievalHitRate: number;
contextOverlap: number;
feedback?: {
comprehension: 'understood' | 'partial' | 'confused';
difficulty: 'easy' | 'appropriate' | 'hard';
clarity: 'confusing' | 'ok' | 'clear';
comment?: string;
timestamp: Timestamp;
};
createdAt: Timestamp;
metadata: {
device: string;
sessionId: string;
ipAddress: string;
};
}
```
**auditLogs/{logId}**
```typescript
interface AuditLog {
id: string;
userId: string;
schoolId: string;
action: string;
resource: string;
resourceId: string;
details: {
oldValue?: any;
newValue?: any;
changes?: string[];
};
timestamp: Timestamp;
ipAddress: string;
userAgent: string;
status: 'success' | 'failed';
errorMessage?: string;
}
```
#### Security Rules:
**firestore.rules**
```javascript
rules_version = '2';
service cloud.firestore {
match /databases/{database}/documents {
// Helper functions
function isAuthenticated() {
return request.auth != null;
}
function isSameSchool(schoolId) {
return isAuthenticated() &&
get(/databases/$(database)/documents/users/$(request.auth.uid)).data.schoolId == schoolId;
}
function hasRole(role) {
return isAuthenticated() &&
get(/databases/$(database)/documents/users/$(request.auth.uid)).data.role == role;
}
// Schools collection - admin only
match /schools/{schoolId} {
allow read, write: if hasRole('admin');
}
// Users collection
match /users/{userId} {
allow read: if isAuthenticated() &&
(request.auth.uid == userId || isSameSchool(resource.data.schoolId));
allow write: if isAuthenticated() &&
(request.auth.uid == userId || hasRole('admin'));
allow create: if isAuthenticated() &&
request.auth.uid == userId &&
request.resource.data.role in ['student', 'teacher', 'admin'];
}
// Learning states - students can only access their own
match /learningStates/{studentId} {
allow read: if isAuthenticated() &&
(request.auth.uid == studentId || isSameSchool(resource.data.schoolId));
allow write: if isAuthenticated() &&
(request.auth.uid == studentId || hasRole('teacher') || hasRole('admin'));
}
// Content chunks - authenticated users from same school
match /contentChunks/{chunkId} {
allow read: if isAuthenticated() && isSameSchool(resource.data.schoolId);
allow write: if isAuthenticated() &&
(hasRole('teacher') || hasRole('admin')) &&
isSameSchool(resource.data.schoolId);
}
// Quizzes
match /quizzes/{quizId} {
allow read: if isAuthenticated() && isSameSchool(resource.data.schoolId);
allow write: if isAuthenticated() &&
(request.auth.uid == resource.data.teacherId || hasRole('admin'));
allow create: if isAuthenticated() &&
hasRole('teacher') &&
request.resource.data.teacherId == request.auth.uid;
}
// Quiz attempts
match /quizAttempts/{attemptId} {
allow read: if isAuthenticated() &&
(request.auth.uid == resource.data.studentId || isSameSchool(resource.data.schoolId));
allow write: if isAuthenticated() &&
(request.auth.uid == resource.data.studentId || hasRole('teacher'));
allow create: if isAuthenticated() &&
request.auth.uid == request.resource.data.studentId;
}
// Interactions
match /interactions/{interactionId} {
allow read: if isAuthenticated() &&
(request.auth.uid == resource.data.studentId || hasRole('teacher'));
allow write: if isAuthenticated() &&
(request.auth.uid == resource.data.studentId || hasRole('teacher'));
allow create: if isAuthenticated() &&
request.auth.uid == request.resource.data.studentId;
}
// Audit logs - read only for admins
match /auditLogs/{logId} {
allow read: if hasRole('admin');
allow write: if hasRole('admin');
}
}
}
```
#### Indexes:
**firestore.indexes.json**
```json
{
"indexes": [
{
"collectionGroup": "contentChunks",
"queryScope": "COLLECTION",
"fields": [
{
"fieldPath": "schoolId",
"order": "ASCENDING"
},
{
"fieldPath": "concept",
"order": "ASCENDING"
},
{
"fieldPath": "pedagogy.difficulty",
"order": "ASCENDING"
}
]
},
{
"collectionGroup": "contentChunks",
"queryScope": "COLLECTION",
"fields": [
{
"fieldPath": "schoolId",
"order": "ASCENDING"
},
{
"fieldPath": "subject",
"order": "ASCENDING"
},
{
"fieldPath": "unit",
"order": "ASCENDING"
}
]
},
{
"collectionGroup": "interactions",
"queryScope": "COLLECTION",
"fields": [
{
"fieldPath": "studentId",
"order": "ASCENDING"
},
{
"fieldPath": "createdAt",
"order": "DESCENDING"
}
]
},
{
"collectionGroup": "quizAttempts",
"queryScope": "COLLECTION",
"fields": [
{
"fieldPath": "studentId",
"order": "ASCENDING"
},
{
"fieldPath": "completedAt",
"order": "DESCENDING"
}
]
},
{
"collectionGroup": "quizAttempts",
"queryScope": "COLLECTION",
"fields": [
{
"fieldPath": "quizId",
"order": "ASCENDING"
},
{
"fieldPath": "score",
"order": "DESCENDING"
}
]
}
],
"fieldOverrides": []
}
```
---
### Task 1.3: Cloud Functions Setup
**Priority**: Critical
**Estimated Time**: 6 hours
**Dependencies**: Task 1.2
#### Subtasks:
- [ ] Initialize Cloud Functions project
- [ ] Set up TypeScript configuration
- [ ] Configure environment variables
- [ ] Set up deployment scripts
- [ ] Create function templates
#### Project Structure:
```
functions/
├── src/
│ ├── index.ts
│ ├── config/
│ │ ├── firebase.ts
│ │ ├── llm.ts
│ │ └── database.ts
│ ├── services/
│ │ ├── auth.service.ts
│ │ ├── rag.service.ts
│ │ ├── llm.service.ts
│ │ ├── analytics.service.ts
│ │ └── storage.service.ts
│ ├── middleware/
│ │ ├── auth.middleware.ts
│ │ ├── validation.middleware.ts
│ │ ├── rate-limit.middleware.ts
│ │ └── error.middleware.ts
│ ├── models/
│ │ ├── user.model.ts
│ │ ├── content.model.ts
│ │ ├── quiz.model.ts
│ │ └── interaction.model.ts
│ ├── utils/
│ │ ├── logger.ts
│ │ ├── validators.ts
│ │ ├── helpers.ts
│ │ └── constants.ts
│ └── functions/
│ ├── auth/
│ │ ├── signUp.ts
│ │ ├── signIn.ts
│ │ └── resetPassword.ts
│ ├── content/
│ │ ├── uploadContent.ts
│ │ ├── processContent.ts
│ │ └── searchContent.ts
│ ├── tutor/
│ │ ├── askTutor.ts
│ │ ├── submitFeedback.ts
│ │ └── getRecommendations.ts
│ ├── quiz/
│ │ ├── createQuiz.ts
│ │ ├── submitQuiz.ts
│ │ └── getResults.ts
│ └── analytics/
│ ├── getStudentProgress.ts
│ ├── getClassAnalytics.ts
│ └── getSystemMetrics.ts
├── package.json
├── tsconfig.json
├── .eslintrc.js
└── .env.example
```
#### Configuration Files:
**package.json**
```json
{
"name": "teachit-functions",
"description": "Cloud Functions for AI Study Assistant",
"scripts": {
"build": "tsc",
"build:watch": "tsc --watch",
"serve": "npm run build && firebase emulators:start --only functions",
"shell": "npm run build && firebase functions:shell",
"start": "npm run shell",
"deploy": "firebase deploy --only functions",
"logs": "firebase functions:log"
},
"engines": {
"node": "18"
},
"main": "lib/index.js",
"dependencies": {
"@google-cloud/firestore": "^6.7.0",
"@google-cloud/storage": "^6.11.0",
"firebase-admin": "^11.10.1",
"firebase-functions": "^4.4.1",
"openai": "^4.20.1",
"anthropic": "^0.6.3",
"sentence-transformers": "^0.0.1",
"faiss-node": "^0.5.1",
"pdf-parse": "^1.1.1",
"mammoth": "^1.6.0",
"express": "^4.18.2",
"cors": "^2.8.5",
"helmet": "^7.0.0",
"express-rate-limit": "^6.10.0",
"joi": "^17.9.2",
"winston": "^3.10.0",
"dotenv": "^16.3.1",
"uuid": "^9.0.0",
"bcryptjs": "^2.4.3",
"jsonwebtoken": "^9.0.2"
},
"devDependencies": {
"@types/express": "^4.17.17",
"@types/cors": "^2.8.13",
"@types/uuid": "^9.0.2",
"@types/bcryptjs": "^2.4.2",
"@types/jsonwebtoken": "^9.0.2",
"typescript": "^5.1.6",
"@typescript-eslint/eslint-plugin": "^6.2.1",
"@typescript-eslint/parser": "^6.2.1",
"eslint": "^8.46.0",
"eslint-config-google": "^0.14.0",
"eslint-plugin-import": "^2.28.0",
"firebase-functions-test": "^3.1.0"
},
"private": true
}
```
**tsconfig.json**
```json
{
"compilerOptions": {
"module": "commonjs",
"noImplicitReturns": true,
"noUnusedLocals": true,
"outDir": "lib",
"sourceMap": true,
"strict": true,
"target": "es2017",
"resolveJsonModule": true,
"esModuleInterop": true,
"skipLibCheck": true,
"forceConsistentCasingInFileNames": true
},
"compileOnSave": true,
"include": [
"src"
],
"exclude": [
"node_modules",
"lib"
]
}
```
---
## 🔐 WEEK 3-4: AUTHENTICATION & USER MANAGEMENT
### Task 2.1: Authentication Functions
**Priority**: Critical
**Estimated Time**: 12 hours
**Dependencies**: Task 1.3
#### Subtasks:
- [ ] Implement user registration
- [ ] Implement user login
- [ ] Implement password reset
- [ ] Add email verification
- [ ] Create user profile management
- [ ] Implement role-based access control
#### Implementation:
**src/functions/auth/signUp.ts**
```typescript
import { https, CallableContext } from 'firebase-functions/v1';
import { getFirestore } from 'firebase-admin/firestore';
import { getAuth } from 'firebase-admin/auth';
import { CreateUserRequest, UserResponse } from '../../models/user.model';
import { validateCreateUser } from '../../utils/validators';
import { logger } from '../../utils/logger';
export const signUp = https.onCall(async (data: CreateUserRequest, context: CallableContext) => {
try {
// Validate input
const validation = validateCreateUser(data);
if (!validation.isValid) {
throw new https.HttpsError('invalid-argument', validation.errors.join(', '));
}
// Check if user already exists
const auth = getAuth();
const existingUser = await auth.getUserByEmail(data.email);
if (existingUser) {
throw new https.HttpsError('already-exists', 'User with this email already exists');
}
// Get school and validate
const db = getFirestore();
const schoolDoc = await db.collection('schools').doc(data.schoolId).get();
if (!schoolDoc.exists) {
throw new https.HttpsError('not-found', 'School not found');
}
const school = schoolDoc.data();
if (!school.isActive) {
throw new https.HttpsError('permission-denied', 'School is not active');
}
// Check subscription limits
const usersSnapshot = await db.collection('users')
.where('schoolId', '==', data.schoolId)
.where('isActive', '==', true)
.get();
const studentCount = usersSnapshot.docs.filter(doc => doc.data().role === 'student').length;
const teacherCount = usersSnapshot.docs.filter(doc => doc.data().role === 'teacher').length;
if (data.role === 'student' && studentCount >= school.subscription.maxStudents) {
throw new https.HttpsError('resource-exhausted', 'School has reached maximum student limit');
}
if (data.role === 'teacher' && teacherCount >= school.subscription.maxTeachers) {
throw new https.HttpsError('resource-exhausted', 'School has reached maximum teacher limit');
}
// Create Firebase Auth user
const userRecord = await auth.createUser({
email: data.email,
password: data.password,
displayName: data.profile.name,
emailVerified: false,
});
// Create user document
const userDoc = {
uid: userRecord.uid,
schoolId: data.schoolId,
role: data.role,
email: data.email,
profile: data.profile,
preferences: {
language: school.settings.language || 'en',
notifications: true,
darkMode: false,
learningStyle: data.profile.learningStyle,
},
createdAt: new Date(),
updatedAt: new Date(),
lastLogin: new Date(),
isActive: true,
emailVerified: false,
};
await db.collection('users').doc(userRecord.uid).set(userDoc);
// Create learning state for students
if (data.role === 'student') {
const learningState = {
studentId: userRecord.uid,
schoolId: data.schoolId,
profile: {
name: data.profile.name,
gradeLevel: data.profile.gradeLevel || 10,
subjects: [],
},
conceptStates: {},
spacedRepetition: {
nextReviewDue: [],
algorithm: 'sm2',
},
learningGoals: [],
adaptiveDifficulty: {
currentLevel: 2,
comfortableMin: 1.5,
comfortableMax: 2.8,
lastAdjusted: new Date(),
},
preferences: {
modePreference: 'EXPLANATION',
exampleFrequency: 'high',
hintStyle: 'guided_questions',
feedbackFrequency: 'immediate',
},
metadata: {
updatedAt: new Date(),
totalInteractions: 0,
dailyActiveDays: 0,
},
};
await db.collection('learningStates').doc(userRecord.uid).set(learningState);
}
// Send email verification
await auth.generateEmailVerificationLink(data.email);
// Log audit
await db.collection('auditLogs').add({
userId: userRecord.uid,
schoolId: data.schoolId,
action: 'user_created',
resource: 'users',
resourceId: userRecord.uid,
details: {
role: data.role,
email: data.email,
},
timestamp: new Date(),
ipAddress: context.rawRequest.ip,
userAgent: context.rawRequest.headers['user-agent'],
status: 'success',
});
logger.info(`User created successfully: ${userRecord.uid}`);
return {
success: true,
user: {
uid: userRecord.uid,
email: data.email,
role: data.role,
profile: data.profile,
},
} as UserResponse;
} catch (error) {
logger.error('Error creating user:', error);
if (error instanceof https.HttpsError) {
throw error;
}
throw new https.HttpsError('internal', 'Failed to create user');
}
});
```
**src/functions/auth/signIn.ts**
```typescript
import { https, CallableContext } from 'firebase-functions/v1';
import { getFirestore } from 'firebase-admin/firestore';
import { getAuth } from 'firebase-admin/auth';
import { SignInRequest, UserResponse } from '../../models/user.model';
import { validateSignIn } from '../../utils/validators';
import { logger } from '../../utils/logger';
export const signIn = https.onCall(async (data: SignInRequest, context: CallableContext) => {
try {
// Validate input
const validation = validateSignIn(data);
if (!validation.isValid) {
throw new https.HttpsError('invalid-argument', validation.errors.join(', '));
}
// Authenticate user
const auth = getAuth();
let userRecord;
try {
userRecord = await auth.getUserByEmail(data.email);
} catch (error) {
throw new https.HttpsError('not-found', 'User not found');
}
// Check if user is active
const db = getFirestore();
const userDoc = await db.collection('users').doc(userRecord.uid).get();
if (!userDoc.exists) {
throw new https.HttpsError('not-found', 'User profile not found');
}
const user = userDoc.data();
if (!user.isActive) {
throw new https.HttpsError('permission-denied', 'Account is deactivated');
}
// Update last login
await userDoc.ref.update({
lastLogin: new Date(),
updatedAt: new Date(),
});
// Log audit
await db.collection('auditLogs').add({
userId: userRecord.uid,
schoolId: user.schoolId,
action: 'user_sign_in',
resource: 'users',
resourceId: userRecord.uid,
timestamp: new Date(),
ipAddress: context.rawRequest.ip,
userAgent: context.rawRequest.headers['user-agent'],
status: 'success',
});
logger.info(`User signed in successfully: ${userRecord.uid}`);
return {
success: true,
user: {
uid: userRecord.uid,
email: user.email,
role: user.role,
profile: user.profile,
preferences: user.preferences,
},
} as UserResponse;
} catch (error) {
logger.error('Error signing in user:', error);
if (error instanceof https.HttpsError) {
throw error;
}
throw new https.HttpsError('internal', 'Failed to sign in user');
}
});
```
---
### Task 2.2: User Profile Management
**Priority**: High
**Estimated Time**: 8 hours
**Dependencies**: Task 2.1
#### Subtasks:
- [ ] Implement profile update function
- [ ] Add avatar upload functionality
- [ ] Create preferences management
- [ ] Implement password change
- [ ] Add account deletion
---
## 📚 WEEK 5-6: CONTENT MANAGEMENT
### Task 3.1: Content Upload & Processing
**Priority**: High
**Estimated Time**: 16 hours
**Dependencies**: Task 2.2
#### Subtasks:
- [ ] Implement file upload endpoint
- [ ] Create PDF parsing service
- [ ] Build content chunking algorithm
- [ ] Add content quality validation
- [ ] Implement metadata extraction
- [ ] Create content indexing
#### Implementation:
**src/services/content.service.ts**
```typescript
import { getFirestore } from 'firebase-admin/firestore';
import { getStorage } from 'firebase-admin/storage';
import * as pdfParse from 'pdf-parse';
import * as mammoth from 'mammoth';
import { ContentChunk, ProcessedContent } from '../models/content.model';
import { chunkContent, validateChunk } from '../utils/content-processor';
import { generateEmbedding } from './llm.service';
import { logger } from '../utils/logger';
export class ContentService {
private db = getFirestore();
private storage = getStorage();
async processUploadedFile(
teacherId: string,
schoolId: string,
file: Buffer,
fileName: string,
mimeType: string,
metadata: any
): Promise<ProcessedContent> {
try {
logger.info(`Processing file: ${fileName} for teacher: ${teacherId}`);
// Extract text based on file type
let text: string;
let extractedMetadata: any = {};
switch (mimeType) {
case 'application/pdf':
const pdfData = await pdfParse(file);
text = pdfData.text;
extractedMetadata = {
pageCount: pdfData.numpages,
info: pdfData.info,
};
break;
case 'application/vnd.openxmlformats-officedocument.wordprocessingml.document':
const docxResult = await mammoth.extractRawText({ buffer: file });
text = docxResult.value;
extractedMetadata = {
wordCount: text.split(/\s+/).length,
};
break;
case 'text/plain':
text = file.toString('utf-8');
extractedMetadata = {
wordCount: text.split(/\s+/).length,
};
break;
default:
throw new Error(`Unsupported file type: ${mimeType}`);
}
// Validate extracted content
if (!text || text.trim().length < 100) {
throw new Error('Insufficient content extracted from file');
}
// Process and chunk content
const chunks = await this.chunkAndProcessContent(
text,
teacherId,
schoolId,
fileName,
metadata,
extractedMetadata
);
// Generate embeddings for all chunks
const chunksWithEmbeddings = await Promise.all(
chunks.map(async (chunk) => {
const embedding = await generateEmbedding(chunk.text);
return {
...chunk,
embeddingVectorId: `vec_${chunk.id}`,
embedding,
};
})
);
// Save chunks to Firestore
const batch = this.db.batch();
chunksWithEmbeddings.forEach((chunk) => {
const docRef = this.db.collection('contentChunks').doc(chunk.id);
batch.set(docRef, chunk);
});
await batch.commit();
// Update teacher's content count
await this.db.collection('users').doc(teacherId).update({
'metadata.contentCount': admin.firestore.FieldValue.increment(1),
updatedAt: new Date(),
});
// Log audit
await this.db.collection('auditLogs').add({
userId: teacherId,
schoolId,
action: 'content_uploaded',
resource: 'contentChunks',
details: {
fileName,
chunkCount: chunks.length,
mimeType,
},
timestamp: new Date(),
status: 'success',
});
logger.info(`Successfully processed ${chunks.length} chunks from ${fileName}`);
return {
success: true,
chunkCount: chunks.length,
chunks: chunksWithEmbeddings,
metadata: extractedMetadata,
};
} catch (error) {
logger.error(`Error processing file ${fileName}:`, error);
throw error;
}
}
private async chunkAndProcessContent(
text: string,
teacherId: string,
schoolId: string,
fileName: string,
metadata: any,
extractedMetadata: any
): Promise<ContentChunk[]> {
const chunks: ContentChunk[] = [];
// Detect sections (teacher markers or automatic)
const sections = this.detectSections(text);
for (const section of sections) {
// Split section into smaller chunks
const textChunks = this.splitIntoChunks(section.content, 400, 50);
for (let i = 0; i < textChunks.length; i++) {
const chunkText = textChunks[i];
const chunkId = `chunk_${Date.now()}_${i}`;
// Extract pedagogical metadata
const pedagogy = this.extractPedagogicalMetadata(chunkText, metadata);
// Create chunk object
const chunk: ContentChunk = {
id: chunkId,
text: chunkText,
concept: section.concept || 'General',
subConcept: section.subConcept,
subject: metadata.subject || 'General',
unit: metadata.unit || 'General',
pedagogy: {
bloomLevel: pedagogy.bloomLevel || 2,
difficulty: pedagogy.difficulty || 0.5,
estimatedLearningTimeMinutes: pedagogy.estimatedTime || 15,
abstractLevel: pedagogy.abstractLevel || 'medium',
},
prerequisites: pedagogy.prerequisites || [],
content: {
type: this.detectContentType(chunkText),
},
commonMisconceptions: [],
relatedConcepts: [],
realWorldApplications: [],
source: {
documentId: `doc_${Date.now()}`,
fileName,
page: extractedMetadata.page,
section: section.title,
teacherId,
},
metadata: {
createdAt: new Date(),
lastUpdated: new Date(),
author: teacherId,
version: '1.0',
qualityScore: this.calculateQualityScore(chunkText),
isFlagged: false,
},
tokens: this.countTokens(chunkText),
language: metadata.language || 'en',
};
// Validate chunk
if (validateChunk(chunk)) {
chunks.push(chunk);
} else {
logger.warn(`Chunk ${chunkId} failed validation`);
}
}
}
return chunks;
}
private detectSections(text: string): Array<{
title: string;
concept?: string;
subConcept?: string;
content: string;
}> {
const sections: Array<{
title: string;
concept?: string;
subConcept?: string;
content: string;
}> = [];
// Look for teacher-defined markers
const conceptStartRegex = /\[CONCEPT_START:\s*([^\]]+)\]/g;
const conceptEndRegex = /\[CONCEPT_END\]/g;
const exampleStartRegex = /\[EXAMPLE_START\]/g;
const exampleEndRegex = /\[EXAMPLE_END\]/g;
let currentSection: any = null;
const lines = text.split('\n');
for (const line of lines) {
const conceptMatch = conceptStartRegex.exec(line);
if (conceptMatch) {
// Save previous section if exists
if (currentSection) {
sections.push(currentSection);
}
// Start new section
currentSection = {
title: conceptMatch[1],
concept: conceptMatch[1],
content: '',
};
continue;
}
if (conceptEndRegex.test(line)) {
if (currentSection) {
sections.push(currentSection);
currentSection = null;
}
continue;
}
if (currentSection) {
currentSection.content += line + '\n';
}
}
// Add last section if exists
if (currentSection) {
sections.push(currentSection);
}
// If no sections found, treat entire text as one section
if (sections.length === 0) {
sections.push({
title: 'Content',
content: text,
});
}
return sections;
}
private splitIntoChunks(text: string, maxTokens: number, overlapTokens: number): string[] {
const words = text.split(/\s+/);
const chunks: string[] = [];
let currentChunk: string[] = [];
let currentTokens = 0;
for (let i = 0; i < words.length; i++) {
const word = words[i];
currentChunk.push(word);
currentTokens++;
// Check if we've reached max tokens
if (currentTokens >= maxTokens) {
chunks.push(currentChunk.join(' '));
// Start next chunk with overlap
currentChunk = [];
currentTokens = 0;
// Add overlap words
const overlapStart = Math.max(0, i - overlapTokens + 1);
for (let j = overlapStart; j <= i; j++) {
currentChunk.push(words[j]);
currentTokens++;
}
}
}
// Add remaining words
if (currentChunk.length > 0) {
chunks.push(currentChunk.join(' '));
}
return chunks;
}
private extractPedagogicalMetadata(text: string, metadata: any): any {
// This would use NLP or rule-based extraction
// For MVP, we'll use simple heuristics
const pedagogy = {
bloomLevel: 2, // Default to Understanding
difficulty: 0.5, // Medium difficulty
estimatedTime: 15, // 15 minutes
abstractLevel: 'medium' as const,
prerequisites: [],
};
// Detect Bloom's level from text
if (text.includes('define') || text.includes('identify')) {
pedagogy.bloomLevel = 1; // Remember
} else if (text.includes('analyze') || text.includes('compare')) {
pedagogy.bloomLevel = 4; // Analyze
} else if (text.includes('create') || text.includes('design')) {
pedagogy.bloomLevel = 6; // Create
}
// Detect difficulty from complexity
const sentences = text.split(/[.!?]+/).length;
const avgWordsPerSentence = text.split(/\s+/).length / sentences;
if (avgWordsPerSentence > 20) {
pedagogy.difficulty = 0.8; // High difficulty
} else if (avgWordsPerSentence < 10) {
pedagogy.difficulty = 0.2; // Low difficulty
}
return pedagogy;
}
private detectContentType(text: string): 'explanation' | 'example' | 'exercise' | 'assessment' {
const lowerText = text.toLowerCase();
if (lowerText.includes('example') || lowerText.includes('for example')) {
return 'example';
} else if (lowerText.includes('exercise') || lowerText.includes('practice')) {
return 'exercise';
} else if (lowerText.includes('question') || lowerText.includes('quiz')) {
return 'assessment';
} else {
return 'explanation';
}
}
private calculateQualityScore(text: string): number {
let score = 0.5; // Base score
// Length check
if (text.length > 100 && text.length < 1000) {
score += 0.2;
}
// Grammar check (simplified)
const sentences = text.split(/[.!?]+/);
if (sentences.length > 1) {
score += 0.1;
}
// Example check
if (text.toLowerCase().includes('example')) {
score += 0.1;
}
// Definition check
if (text.toLowerCase().includes('define') || text.toLowerCase().includes('is')) {
score += 0.1;
}
return Math.min(1.0, score);
}
private countTokens(text: string): number {
// Simple token estimation (roughly 4 characters per token)
return Math.ceil(text.length / 4);
}
}
```
---
### Task 3.2: Content Search & Retrieval
**Priority**: High
**Estimated Time**: 12 hours
**Dependencies**: Task 3.1
#### Subtasks:
- [ ] Implement keyword search (BM25)
- [ ] Create vector similarity search
- [ ] Build hybrid retrieval engine
- [ ] Add metadata filtering
- [ ] Implement ranking algorithm
- [ ] Create content recommendations
---
## 🤖 WEEK 7-8: AI TUTOR INTEGRATION
### Task 4.1: RAG Pipeline Implementation
**Priority**: High
**Estimated Time**: 20 hours
**Dependencies**: Task 3.2
#### Subtasks:
- [ ] Implement query understanding
- [ ] Create context retrieval
- [ ] Build prompt assembly
- [ ] Integrate LLM API
- [ ] Add hallucination detection
- [ ] Implement response filtering
#### Implementation:
**src/services/rag.service.ts**
```typescript
import { getFirestore } from 'firebase-admin/firestore';
import { ContentChunk, RetrievalResult, RAGContext } from '../models/content.model';
import { ContentService } from './content.service';
import { LLMService } from './llm.service';
import { logger } from '../utils/logger';
export class RAGService {
private db = getFirestore();
private contentService = new ContentService();
private llmService = new LLMService();
async processStudentQuery(
studentId: string,
query: string,
mode: string = 'EXPLANATION'
): Promise<any> {
try {
logger.info(`Processing query for student ${studentId}: "${query}"`);
// Get student learning state
const learningStateDoc = await this.db
.collection('learningStates')
.doc(studentId)
.get();
if (!learningStateDoc.exists) {
throw new Error('Student learning state not found');
}
const learningState = learningStateDoc.data();
// Step 1: Query Understanding
const queryAnalysis = await this.analyzeQuery(query, learningState);
// Step 2: Context Retrieval
const retrievalResult = await this.retrieveContext(
query,
queryAnalysis,
learningState
);
// Step 3: Check if we have sufficient context
if (retrievalResult.chunks.length === 0) {
return this.handleNoContext(query, queryAnalysis);
}
// Step 4: Build RAG Context
const ragContext = this.buildRAGContext(
query,
queryAnalysis,
retrievalResult,
learningState,
mode
);
// Step 5: Generate Response
const response = await this.llmService.generateResponse(ragContext);
// Step 6: Post-processing
const processedResponse = await this.postProcessResponse(
response,
retrievalResult,
ragContext
);
// Step 7: Log Interaction
await this.logInteraction(
studentId,
query,
processedResponse,
retrievalResult,
ragContext
);
return processedResponse;
} catch (error) {
logger.error('Error processing student query:', error);
throw error;
}
}
private async analyzeQuery(query: string, learningState: any): Promise<any> {
const analysis = {
intent: this.detectIntent(query),
studentLevel: learningState.adaptiveDifficulty.currentLevel,
detectedSubject: this.detectSubject(query),
detectedConcept: this.detectConcept(query),
queryComplexity: this.assessComplexity(query),
expandedQuery: this.expandQuery(query),
};
logger.info('Query analysis:', analysis);
return analysis;
}
private detectIntent(query: string): string {
const lowerQuery = query.toLowerCase();
if (lowerQuery.includes('what is') || lowerQuery.includes('define')) {
return 'ask_concept';
} else if (lowerQuery.includes('how to') || lowerQuery.includes('solve')) {
return 'solve_problem';
} else if (lowerQuery.includes('why') || lowerQuery.includes('explain')) {
return 'explain_why';
} else if (lowerQuery.includes('example') || lowerQuery.includes('show me')) {
return 'request_example';
} else {
return 'general_question';
}
}
private detectSubject(query: string): string {
const subjects = ['math', 'calculus', 'algebra', 'geometry', 'physics', 'chemistry'];
const lowerQuery = query.toLowerCase();
for (const subject of subjects) {
if (lowerQuery.includes(subject)) {
return subject;
}
}
return 'general';
}
private detectConcept(query: string): string {
// This would use a more sophisticated approach in production
// For MVP, we'll use simple keyword matching
const concepts = [
'derivative', 'integral', 'limit', 'function', 'equation',
'velocity', 'acceleration', 'force', 'energy'
];
const lowerQuery = query.toLowerCase();
for (const concept of concepts) {
if (lowerQuery.includes(concept)) {
return concept;
}
}
return 'unknown';
}
private assessComplexity(query: string): number {
const words = query.split(/\s+/);
const sentences = query.split(/[.!?]+/).length;
const avgWordsPerSentence = words.length / sentences;
let complexity = 0.3; // Base complexity
if (avgWordsPerSentence > 15) complexity += 0.2;
if (words.length > 20) complexity += 0.2;
if (query.includes('why') || query.includes('explain')) complexity += 0.1;
if (query.includes('compare') || query.includes('analyze')) complexity += 0.2;
return Math.min(1.0, complexity);
}
private expandQuery(query: string): string[] {
// Simple query expansion using synonyms
const expansions: string[] = [query];
const synonyms: { [key: string]: string[] } = {
'derivative': ['rate of change', 'slope', 'differentiation'],
'integral': ['antiderivative', 'integration', 'area under curve'],
'function': ['mapping', 'relation', 'transformation'],
};
const lowerQuery = query.toLowerCase();
for (const [term, syns] of Object.entries(synonyms)) {
if (lowerQuery.includes(term)) {
for (const synonym of syns) {
expansions.push(query.replace(new RegExp(term, 'gi'), synonym));
}
}
}
return expansions;
}
private async retrieveContext(
query: string,
queryAnalysis: any,
learningState: any
): Promise<RetrievalResult> {
const retrievalStrategies = [
this.keywordSearch.bind(this),
this.vectorSearch.bind(this),
this.metadataSearch.bind(this),
];
const allResults: ContentChunk[] = [];
const strategyResults: { [strategy: string]: ContentChunk[] } = {};
// Run all retrieval strategies
for (const strategy of retrievalStrategies) {
try {
const results = await strategy(query, queryAnalysis, learningState);
strategyResults[strategy.name] = results;
allResults.push(...results);
} catch (error) {
logger.error(`Error in ${strategy.name}:`, error);
}
}
// Deduplicate and rank results
const uniqueResults = this.deduplicateResults(allResults);
const rankedResults = this.rankResults(uniqueResults, query, queryAnalysis);
// Filter by difficulty level
const filteredResults = rankedResults.filter(
chunk => chunk.pedagogy.difficulty <= learningState.adaptiveDifficulty.currentLevel / 6
);
// Take top results
const topResults = filteredResults.slice(0, 5);
return {
chunks: topResults,
totalFound: allResults.length,
strategyResults,
queryAnalysis,
};
}
private async keywordSearch(
query: string,
queryAnalysis: any,
learningState: any
): Promise<ContentChunk[]> {
const expandedQueries = queryAnalysis.expandedQuery;
const results: ContentChunk[] = [];
for (const expandedQuery of expandedQueries) {
const snapshot = await this.db
.collection('contentChunks')
.where('schoolId', '==', learningState.schoolId)
.where('text', 'array-contains', expandedQuery.split(/\s+/)[0])
.limit(10)
.get();
snapshot.docs.forEach(doc => {
const chunk = doc.data() as ContentChunk;
chunk.id = doc.id;
results.push(chunk);
});
}
return results;
}
private async vectorSearch(
query: string,
queryAnalysis: any,
learningState: any
): Promise<ContentChunk[]> {
// This would integrate with a vector database (FAISS, Weaviate, etc.)
// For MVP, we'll use a simplified approach
const queryEmbedding = await this.llmService.generateEmbedding(query);
// In production, this would query the vector database
// For now, return empty results
return [];
}
private async metadataSearch(
query: string,
queryAnalysis: any,
learningState: any
): Promise<ContentChunk[]> {
let queryBuilder = this.db
.collection('contentChunks')
.where('schoolId', '==', learningState.schoolId);
// Filter by subject if detected
if (queryAnalysis.detectedSubject !== 'general') {
queryBuilder = queryBuilder.where('subject', '==', queryAnalysis.detectedSubject);
}
// Filter by concept if detected
if (queryAnalysis.detectedConcept !== 'unknown') {
queryBuilder = queryBuilder.where('concept', '==', queryAnalysis.detectedConcept);
}
const snapshot = await queryBuilder.limit(10).get();
return snapshot.docs.map(doc => {
const chunk = doc.data() as ContentChunk;
chunk.id = doc.id;
return chunk;
});
}
private deduplicateResults(results: ContentChunk[]): ContentChunk[] {
const seen = new Set<string>();
return results.filter(chunk => {
const key = `${chunk.concept}_${chunk.text.substring(0, 50)}`;
if (seen.has(key)) {
return false;
}
seen.add(key);
return true;
});
}
private rankResults(
results: ContentChunk[],
query: string,
queryAnalysis: any
): ContentChunk[] {
return results
.map(chunk => ({
...chunk,
score: this.calculateRelevanceScore(chunk, query, queryAnalysis),
}))
.sort((a, b) => b.score - a.score);
}
private calculateRelevanceScore(
chunk: ContentChunk,
query: string,
queryAnalysis: any
): number {
let score = 0;
// Text similarity (simplified)
const queryWords = query.toLowerCase().split(/\s+/);
const chunkWords = chunk.text.toLowerCase().split(/\s+/);
const commonWords = queryWords.filter(word => chunkWords.includes(word));
score += (commonWords.length / queryWords.length) * 0.4;
// Concept matching
if (chunk.concept.toLowerCase().includes(queryAnalysis.detectedConcept.toLowerCase())) {
score += 0.3;
}
// Subject matching
if (chunk.subject.toLowerCase().includes(queryAnalysis.detectedSubject.toLowerCase())) {
score += 0.2;
}
// Difficulty appropriateness
const difficultyDiff = Math.abs(chunk.pedagogy.difficulty - queryAnalysis.studentLevel / 6);
score += (1 - difficultyDiff) * 0.1;
return score;
}
private buildRAGContext(
query: string,
queryAnalysis: any,
retrievalResult: RetrievalResult,
learningState: any,
mode: string
): RAGContext {
const contextText = retrievalResult.chunks
.map(chunk => `[${chunk.concept}]\n${chunk.text}`)
.join('\n\n');
return {
query,
mode,
studentLevel: learningState.adaptiveDifficulty.currentLevel,
contextText,
retrievedChunks: retrievalResult.chunks,
queryAnalysis,
learningState,
constraints: this.getModeConstraints(mode, learningState),
};
}
private getModeConstraints(mode: string, learningState: any): any {
const baseConstraints = {
maxTokens: 500,
temperature: 0.7,
includeExamples: true,
avoidProofs: learningState.adaptiveDifficulty.currentLevel < 4,
};
switch (mode) {
case 'EXPLANATION':
return {
...baseConstraints,
bloomLevel: Math.min(learningState.adaptiveDifficulty.currentLevel, 3),
style: 'explanatory',
includeExamples: true,
};
case 'TUTOR':
return {
...baseConstraints,
bloomLevel: learningState.adaptiveDifficulty.currentLevel,
style: 'socratic',
askQuestions: true,
progressiveHints: true,
};
case 'EXAM':
return {
...baseConstraints,
bloomLevel: 'any',
style: 'formal',
includeExamples: false,
giveAnswers: false,
};
case 'QUIZ':
return {
...baseConstraints,
bloomLevel: Math.min(learningState.adaptiveDifficulty.currentLevel, 2),
style: 'interactive',
immediateFeedback: true,
};
default:
return baseConstraints;
}
}
private async handleNoContext(query: string, queryAnalysis: any): Promise<any> {
return {
status: 'no_context',
message: 'Sorry, I don\'t have content on that topic yet.',
suggestions: await this.generateSuggestions(query, queryAnalysis),
fallbackMode: 'partial_with_hint',
};
}
private async generateSuggestions(query: string, queryAnalysis: any): Promise<string[]> {
const suggestions: string[] = [];
// Suggest learning prerequisites
if (queryAnalysis.detectedConcept !== 'unknown') {
suggestions.push(`Would you like to learn about prerequisites for ${queryAnalysis.detectedConcept}?`);
}
// Suggest related topics
suggestions.push('Try asking about basic concepts first.');
suggestions.push('Would you like me to help you with a different topic?');
return suggestions;
}
private async postProcessResponse(
response: any,
retrievalResult: RetrievalResult,
ragContext: RAGContext
): Promise<any> {
// Detect hallucination risk
const hallucinationScore = this.detectHallucinationRisk(
response.text,
retrievalResult.chunks
);
// Filter inappropriate content
const filteredResponse = this.filterResponse(response.text, ragContext.constraints);
return {
...response,
text: filteredResponse,
hallucinationScore,
retrievalHitRate: retrievalResult.chunks.length > 0 ? 1 : 0,
contextOverlap: this.calculateContextOverlap(response.text, retrievalResult.chunks),
metadata: {
chunksUsed: retrievalResult.chunks.length,
mode: ragContext.mode,
studentLevel: ragContext.studentLevel,
},
};
}
private detectHallucinationRisk(response: string, chunks: ContentChunk[]): number {
// Simple hallucination detection
const responseWords = response.toLowerCase().split(/\s+/);
const contextWords = chunks
.map(chunk => chunk.text.toLowerCase())
.join(' ')
.split(/\s+/);
const commonWords = responseWords.filter(word => contextWords.includes(word));
const overlap = commonWords.length / responseWords.length;
return 1 - overlap; // Higher score = higher risk
}
private filterResponse(response: string, constraints: any): string {
let filtered = response;
// Remove content that violates constraints
if (constraints.avoidProofs) {
filtered = filtered.replace(/proof|prove|theorem/gi, '[proof omitted]');
}
// Ensure response length
if (constraints.maxTokens && filtered.length > constraints.maxTokens * 4) {
filtered = filtered.substring(0, constraints.maxTokens * 4) + '...';
}
return filtered;
}
private calculateContextOverlap(response: string, chunks: ContentChunk[]): number {
const responseWords = response.toLowerCase().split(/\s+/);
const contextWords = chunks
.map(chunk => chunk.text.toLowerCase())
.join(' ')
.split(/\s+/);
const commonWords = responseWords.filter(word => contextWords.includes(word));
return commonWords.length / responseWords.length;
}
private async logInteraction(
studentId: string,
query: string,
response: any,
retrievalResult: RetrievalResult,
ragContext: RAGContext
): Promise<void> {
await this.db.collection('interactions').add({
studentId,
schoolId: ragContext.learningState.schoolId,
type: 'question',
query,
response: response.text,
mode: ragContext.mode,
retrievedChunks: retrievalResult.chunks.map(chunk => ({
chunkId: chunk.id,
confidence: chunk.score || 0.5,
relevanceScore: chunk.score || 0.5,
})),
llmMetadata: {
promptTokens: response.promptTokens || 0,
completionTokens: response.completionTokens || 0,
totalTokens: response.totalTokens || 0,
latencyMs: response.latencyMs || 0,
model: response.model || 'unknown',
temperature: ragContext.constraints.temperature,
},
hallucinationScore: response.hallucinationScore || 0,
retrievalHitRate: retrievalResult.chunks.length > 0 ? 1 : 0,
contextOverlap: response.contextOverlap || 0,
createdAt: new Date(),
});
}
}
```
---
### Task 4.2: LLM Integration
**Priority**: High
**Estimated Time**: 12 hours
**Dependencies**: Task 4.1
#### Subtasks:
- [ ] Set up OpenAI/Anthropic API
- [ ] Implement prompt engineering
- [ ] Create response generation
- [ ] Add token counting
- [ ] Implement rate limiting
- [ ] Add error handling
---
## 📊 WEEK 9-10: QUIZ SYSTEM & ANALYTICS
### Task 5.1: Quiz Management
**Priority**: High
**Estimated Time**: 16 hours
**Dependencies**: Task 4.2
#### Subtasks:
- [ ] Create quiz creation endpoint
- [ ] Implement quiz taking logic
- [ ] Add automatic grading
- [ ] Build quiz analytics
- [ ] Create quiz history
- [ ] Implement quiz recommendations
---
### Task 5.2: Analytics Engine
**Priority**: Medium
**Estimated Time**: 12 hours
**Dependencies**: Task 5.1
#### Subtasks:
- [ ] Implement student progress tracking
- [ ] Create class analytics
- [ ] Build learning recommendations
- [ ] Add performance metrics
- [ ] Create analytics dashboard data
- [ ] Implement export functionality
---
## 🧪 WEEK 11-12: TESTING & DEPLOYMENT
### Task 6.1: Testing Suite
**Priority**: High
**Estimated Time**: 16 hours
**Dependencies**: All previous tasks
#### Subtasks:
- [ ] Write unit tests for all services
- [ ] Create integration tests
- [ ] Test Firebase security rules
- [ ] Load testing for APIs
- [ ] Error scenario testing
- [ ] Performance benchmarking
---
### Task 6.2: Production Deployment
**Priority**: High
**Estimated Time**: 8 hours
**Dependencies**: Task 6.1
#### Subtasks:
- [ ] Configure production environment
- [ ] Set up monitoring and logging
- [ ] Configure alerts
- [ ] Deploy to production
- [ ] Run smoke tests
- [ ] Monitor performance
---
## 📋 DELIVERABLES
### Week 2 Deliverables
- [ ] Firebase project with complete configuration
- [ ] Firestore database with all collections
- [ ] Security rules and indexes
- [ ] Cloud Functions project structure
### Week 4 Deliverables
- [ ] Complete authentication system
- [ ] User management functions
- [ ] Role-based access control
- [ ] Profile management
### Week 6 Deliverables
- [ ] Content upload and processing
- [ ] Content search and retrieval
- [ ] Content management system
- [ ] Quality validation
### Week 8 Deliverables
- [ ] Complete RAG pipeline
- [ ] LLM integration
- [ ] AI tutor functionality
- [ ] Response generation and filtering
### Week 10 Deliverables
- [ ] Quiz system
- [ ] Analytics engine
- [ ] Progress tracking
- [ ] Recommendation system
### Week 12 Deliverables
- [ ] Complete test suite
- [ ] Production deployment
- [ ] Monitoring and logging
- [ ] Documentation
---
## 🔧 ENVIRONMENT CONFIGURATION
### Development Environment
```bash
# Firebase emulators
firebase emulators:start
# Local functions
npm run serve
# Test data seeding
npm run seed:data
```
### Production Environment
```bash
# Deploy all functions
firebase deploy --only functions
# Deploy specific function
firebase deploy --only functions:askTutor
# View logs
firebase functions:log
```
---
## 📈 MONITORING & LOGGING
### Key Metrics to Track
- API response times
- Error rates
- LLM token usage
- Retrieval hit rates
- User engagement
- System performance
### Alert Configuration
- High error rates (>5%)
- Slow responses (>3s)
- LLM quota exhaustion
- Database connection issues
- Storage capacity warnings
---
## 🛡️ SECURITY CONSIDERATIONS
### API Security
- Rate limiting per user
- Input validation and sanitization
- SQL injection prevention
- XSS protection
- CORS configuration
### Data Security
- Encryption at rest and in transit
- Access control validation
- Audit logging
- Data retention policies
- GDPR compliance
---
*Last Updated: 2026-05-06*
*Version: 1.0.0*
*Backend Lead: Cloud Functions Development Team*