240405/LearnIT
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

.md

Browse Source
This commit is contained in:
240403
2026-05-25 21:41:41 +01:00
parent c2fd663170
commit 3d3747d3a2
13 changed files with 426 additions and 337 deletions
Download Patch File Download Diff File Expand all files Collapse all files

365
docs/ARCHITECTURE.md
View File

@@ -1,6 +1,8 @@
# Architecture Overview - AI Study Assistant
## 🏗️ COMPLETE SYSTEM ARCHITECTURE
## 🏗️ ACTUAL SYSTEM ARCHITECTURE
> ⚠️ **Nota importante**: Esta documentação reflete a arquitetura REAL implementada no código. O projeto é uma aplicação Flutter que comunica diretamente com Firebase e Ollama, sem backend Node.js ou Python intermediário.
---
@@ -31,46 +33,45 @@ This document provides a comprehensive overview of the AI Study Assistant system
## 🏛️ HIGH-LEVEL ARCHITECTURE
### System Overview
### System Overview (Real Implementation)
```
┌─────────────────────────────────────────────────────────────────┐
│ PRESENTATION LAYER │
├─────────────────┬─────────────────┬─────────────────────────────┤
│ Flutter App │ Web App │ Admin Dashboard │
│ (Mobile/Web) │ (PWA) │ (Management) │
│ Flutter App │ Web App │ (Same codebase) │
│ (Mobile/Web) │ (Flutter Web) │ │
└─────────────────┴─────────────────┴─────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ API GATEWAY │
│ • Authentication • Rate Limiting • Load Balancing │
└─────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ SERVICE LAYER │
│ FLUTTER SERVICES │
├─────────────┬─────────────┬─────────────┬───────────────────────┤
│ Auth │ Tutor │ Quiz │ Analytics │
│ Auth │ Tutor │ Quiz │ Gamification │
│ Service │ Service │ Service │ Service │
├─────────────┼─────────────┼─────────────┼───────────────────────┤
│ RAG │ Chat │ Content │ Vector │
│ Service │ Memory │ Service │ Service (Mock) │
└─────────────┴─────────────┴─────────────┴───────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ AI/ML LAYER │
├─────────────┬─────────────┬─────────────┬───────────────────────┤
│ RAG │ Embedding │ LLM │ Vector Store │
│ Engine │ Service │ Service │ (FAISS) │
└─────────────┴─────────────┴─────────────┴───────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ DATA LAYER │
├─────────────┬─────────────┬─────────────┬───────────────────────┤
│ Firestore │ Storage │ Cache │ Search │
│ Database │ (Files) │ (Redis) │ (Elasticsearch) │
└─────────────┴─────────────┴─────────────┴───────────────────────┘
│ EXTERNAL SERVICES │
├─────────────────────────┬─────────────────────────────────────┤
│ Firebase │ Ollama LLM │
│ • Auth │ • Model: qwen3-coder:30b │
│ • Firestore │ • Endpoint: /api/chat │
│ • Storage │ │
│ • Analytics │ │
│ • Crashlytics │ │
└─────────────────────────┴─────────────────────────────────────┘
```
### Architecture Notes
- **No Backend Server**: The Flutter app communicates directly with Firebase and Ollama
- **RAG Implementation**: Keyword-based search with windowing, implemented in Dart
- **Embeddings**: Mock/simulated embeddings (384 dimensions) using text hashing
- **Vector Store**: Not FAISS - simple in-memory Firestore storage with cosine similarity
---
## 📱 FRONTEND ARCHITECTURE
@@ -187,7 +188,14 @@ class TutorRepositoryImpl implements TutorRepository {
## ⚡ BACKEND ARCHITECTURE
### Cloud Functions Architecture
> ⚠️ **Nota**: Não existe backend Node.js/Cloud Functions. A arquitetura descrita abaixo é para referência futura apenas.
### Actual Backend
The "backend" consists of:
1. **Firebase Services** (managed by Google)
2. **Ollama Instance** (self-hosted at 89.114.196.110:11434)
### Cloud Functions Architecture (NOT IMPLEMENTED)
```
┌─────────────────────────────────────────────────────────────────┐
│ API GATEWAY LAYER │
@@ -198,6 +206,8 @@ class TutorRepositoryImpl implements TutorRepository {
│ • CORS │ • Schema Valid │ • Per-Endpoint Limits │
│ • Logging │ • Sanitization │ • Global Limits │
│ • Error Handl │ • Type Check │ • Burst Protection │
Note: This layer does not exist in the current implementation.
└─────────────────┴─────────────────┴─────────────────────────────┘
│
▼
@@ -274,32 +284,32 @@ export class TutorService {
---
## 🤖 AI/ML ARCHITECTURE
## 🤖 AI/ML ARCHITECTURE (DART IMPLEMENTATION)
### RAG Engine Architecture
### RAG Engine Architecture (Real Implementation)
```
┌─────────────────────────────────────────────────────────────────┐
│ INPUT PROCESSING │
│ INPUT PROCESSING (Dart) │
├─────────────────┬─────────────────┬─────────────────────────────┤
│ Text Input │ Content │ Query Processing │
│ Processing │ Processing │ │
│ │ │ │
│ • Tokenization │ • PDF Parsing │ • Query Embedding │
│ • Cleaning │ • Text Extract │ • Vector Search │
│ • Normalization │ • Chunking │ • Similarity Calculation │
│ • Validation │ • Metadata │ • Ranking │
│ • Text Cleaning │ • PDF Parsing │ • Keyword Extraction │
│ • LaTeX Filter │ • Text Extract │ • Window Search │
│ • Normalization │ • Chunking │ • Similarity Matching │
│ • Validation │ • Cache (Hive) │ • Content Ranking │
└─────────────────┴─────────────────┴─────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ VECTOR STORE │
│ VECTOR STORE (Mock/Simple) │
├─────────────────┬─────────────────┬─────────────────────────────┤
│ Indexing │ Storage │ Retrieval │
│ Embeddings │ Storage │ Retrieval │
│ │ │ │
│ • FAISS Index │ • Vector Data │ • Approximate Search │
│ • HNSW Tree │ • Metadata │ • Exact Search │
│ • IVF Clusters │ • Chunks │ • Hybrid Search │
│ • Optimization │ • Updates │ • Filtering │
│ • Hash-based │ • Firestore │ • Cosine Similarity │
│ • 384 dims │ • contentChunks │ • Keyword Matching │
│ • Deterministic │ • materials │ • Window-based Search │
│ • Fast/Cheap │ • No FAISS │ • No Approximate Search │
└─────────────────┴─────────────────┴─────────────────────────────┘
│
▼
@@ -309,107 +319,124 @@ export class TutorService {
│ Prompt │ Generation │ Post-Processing │
│ Engineering │ │ │
│ │ │ │
│ • Context Build │ • OpenAI API │ • Response Validation │
│ • Template │ • Anthropic API│ • Safety Checks │
│ • Formatting │ • Model Selection│ • Quality Assessment │
│ • Safety │ • Rate Limit │ • Caching │
│ • Context Embed │ • Ollama API │ • UTF-8 Encoding │
│ • User Persona │ • qwen3-coder │ • Response Formatting │
│ • PDF Content │ • 30B model │ • Citation Handling │
│ • Constraints │ • Direct HTTP │ • Error Handling │
└─────────────────┴─────────────────┴─────────────────────────────┘
```
### RAG Pipeline Implementation
```python
# RAG Engine Pipeline
class RAGPipeline:
def __init__(self):
self.embedding_service = EmbeddingService()
self.vector_store = VectorStore()
self.llm_service = LLMService()
self.prompt_builder = PromptBuilder()
async def process_query(self, query: str, mode: str = "EXPLANATION") -> str:
# Step 1: Process input
processed_query = self.preprocess_query(query)
# Step 2: Generate embedding
query_embedding = await self.embedding_service.encode([processed_query])
# Step 3: Retrieve relevant context
context_chunks = await self.vector_store.search(
query_embedding[0],
k=10,
filters=self.get_filters(mode)
)
# Step 4: Build prompt
prompt = self.prompt_builder.build(
query=processed_query,
context=context_chunks,
mode=mode
)
# Step 5: Generate response
response = await self.llm_service.generate(prompt)
# Step 6: Post-process
final_response = self.postprocess_response(response, context_chunks)
return final_response
### RAG Pipeline Implementation (Dart)
```dart
// Real implementation in lib/core/services/rag_ai_service.dart
class RAGAIService {
static const String _baseUrl = 'http://89.114.196.110:11434/api/chat';
static const String _model = 'qwen3-coder:30b';
static Future<String> askTutor({
required String question,
required List<String> materialIds,
required String mode,
}) async {
// Step 1: Retrieve context from materials (keyword-based)
final context = await MaterialsRAGService.getContextForQuestion(
question: question,
materialIds: materialIds,
);
def preprocess_query(self, query: str) -> str:
# Clean and normalize query
query = query.strip().lower()
# Remove special characters
query = re.sub(r'[^\w\s]', '', query)
# Tokenize and normalize
return query
// Step 2: Build prompt with embedded context
final prompt = _buildPrompt(
question: question,
context: context,
mode: mode,
);
def get_filters(self, mode: str) -> Dict[str, Any]:
# Mode-specific filtering
filters = {}
if mode == "EXPLANATION":
filters["content_type"] = ["explanation", "definition"]
elif mode == "TUTOR":
filters["difficulty"] = {"$lte": 0.7}
return filters
// Step 3: Call Ollama API directly
final response = await http.post(
Uri.parse(_baseUrl),
headers: {'Content-Type': 'application/json; charset=utf-8'},
body: utf8.encode(jsonEncode({
'model': _model,
'messages': [
{'role': 'system', 'content': _systemPrompt},
{'role': 'user', 'content': prompt},
],
})),
);
// Step 4: Process response
return _processResponse(response);
}
}
// MaterialsRAGService - keyword window search
class MaterialsRAGService {
static Future<String> getContextForQuestion({
required String question,
required List<String> materialIds,
}) async {
// PDF extraction with syncfusion_flutter_pdf
// Keyword matching with windowing (1200 chars)
// No FAISS, no embeddings, no vector search
}
}
```
### Key Differences from Original Design
- ❌ **No Python RAG Engine**: Implemented entirely in Dart
- ❌ **No FAISS**: Uses keyword matching and simple cosine similarity
- ❌ **No Sentence Transformers**: Hash-based mock embeddings (384 dims)
- ❌ **No OpenAI/Anthropic**: Only Ollama (qwen3-coder:30b)
- ✅ **PDF Processing**: syncfusion_flutter_pdf for text extraction
- ✅ **Caching**: Hive for PDF content caching
- ✅ **Firestore**: Stores contentChunks with simple vector data
---
## 🗄️ DATA ARCHITECTURE
### Database Schema
### Database Schema (Actual Implementation)
```
┌─────────────────────────────────────────────────────────────────┐
│ FIRESTORE DATABASE │
├─────────────────┬─────────────────┬─────────────────────────────┤
│ USERS │ CONTENT │ LEARNING │
│ USERS │ MATERIALS │ CONTENTCHUNKS │
│ │ │ │
│ • uid │ • id │ • studentId │
│ • email │ • title │ • concept │
│ • role │ • subject │ • mastery │
│ • schoolId │ • concept │ • confidence │
│ • profile │ • difficulty │ • lastInteraction │
│ • preferences │ • grade │ • interactions │
│ • createdAt │ • uploadedAt │ • recommendations │
│ • lastActive │ • uploadedBy │ • spacedRepetition │
│ • uid │ • id │ • id │
│ • email │ • teacherId │ • text │
│ • role │ • fileName │ • subject │
│ • schoolId │ • fileUrl │ • concept │
│ • displayName │ • type │ • embedding (List<double>) │
│ • createdAt │ • createdAt │ • metadata │
│ │ │ • createdAt │
└─────────────────┴─────────────────┴─────────────────────────────┘
│
▼
┌─────────────────┬─────────────────┬─────────────────────────────┐
│ QUIZ │ INTERACTIONS │ SCHOOLS │
│ │ │ │
│ • id │ • id │ • id │
│ • title │ • studentId │ • name │
│ • subject │ • query │ • email │
│ • concept │ • response │ • settings │
│ • questions │ • mode │ • subscription │
│ • timeLimit │ • timestamp │ • maxStudents │
│ • passingScore │ • feedback │ • maxTeachers │
│ • createdBy │ • metadata │ • isActive │
│ • createdAt │ • sources │ • createdAt │
│ QUIZZES │ CONVERSATIONS │ CLASSES/ENROLLMENTS │
│ │ (userChats/*) │ │
│ • id │ │ • id │
│ • teacherId │ • id │ • teacherId/classId │
│ • title │ • title │ • name/code │
│ • description │ • selectedMaterials│ • studentId │
│ • questions │ • createdAt │ • createdAt/joinedAt │
│ • createdAt │ • hasUserMessage│ │
│ │ • messages/* │ │
└─────────────────┴─────────────────┴─────────────────────────────┘
```
### Collections NOT Implemented
- ❌ `learningStates` - Not in codebase
- ❌ `auditLogs` - Not implemented
- ❌ `quizAttempts` - Not implemented
- ❌ `interactions` - Replaced by userChats/{uid}/conversations
### Roles (Only 2 implemented)
- ✅ `student` - Can view content, take quizzes, ask tutor
- ✅ `teacher` - Can upload materials, create quizzes, view analytics
- ❌ `admin` - NOT IMPLEMENTED
- ❌ `super_admin` - NOT IMPLEMENTED
### Data Flow Architecture
```
┌─────────────────────────────────────────────────────────────────┐
@@ -524,88 +551,84 @@ class RAGPipeline:
---
## 🔧 TECHNOLOGY STACK
## 🔧 TECHNOLOGY STACK (ACTUAL)
### Frontend Technologies
```yaml
Flutter Framework:
- SDK: 3.41.0+
- SDK: ^3.11.5 (Dart 3.0+)
- Language: Dart 3.0+
- State Management: Riverpod 2.4.9
- Navigation: GoRouter 12.1.3
- UI: Material Design 3
- Testing: Flutter Test, Integration Test
- Testing: Flutter Test, Integration Test (not implemented)
Firebase Services:
- Authentication: Firebase Auth
- Database: Cloud Firestore
- Storage: Firebase Storage
- Analytics: Firebase Analytics
- Crashlytics: Firebase Crashlytics
- Performance: Firebase Performance
- Authentication: firebase_auth ^4.17.8
- Database: cloud_firestore ^4.15.8
- Storage: firebase_storage ^11.6.9
- Analytics: firebase_analytics ^10.8.0
- Crashlytics: firebase_crashlytics ^3.5.7
- Messaging: firebase_messaging ^14.9.3
- Performance: NOT IMPLEMENTED
- Remote Config: NOT IMPLEMENTED
Third-Party Libraries:
- HTTP: Dio 5.4.0
- Caching: Cached Network Image 3.3.0
- Fonts: Google Fonts 6.1.0
- Animations: Flutter Animate 4.2.0
- Local Storage: Hive 2.2.3
- HTTP: Dio ^5.4.0, http ^1.1.2
- PDF Processing: syncfusion_flutter_pdf ^33.2.6
- Caching: cached_network_image ^3.3.0, hive ^2.2.3
- Fonts: google_fonts ^6.1.0
- Animations: flutter_animate ^4.2.0, lottie ^2.7.0
- Charts: fl_chart ^0.64.0
- File Handling: file_selector ^1.0.3, image_picker ^1.0.4
- Utilities: intl ^0.20.2, uuid ^4.2.1, equatable ^2.0.5
```
### Backend Technologies
### Backend Technologies (NOT IMPLEMENTED)
```yaml
Runtime Environment:
- Platform: Firebase Cloud Functions
- Runtime: Node.js 18.x LTS
- Language: TypeScript 5.0+
- Package Manager: npm 9.x
Status: NO BACKEND SERVER
Core Services:
- Authentication: Firebase Admin SDK
- Database: Firestore Admin SDK
- Storage: Cloud Storage Admin SDK
- HTTP Framework: Express.js 4.18+
- Validation: Joi 17.9+
- Security: Helmet 7.0+
The following technologies are documented but NOT implemented:
❌ Firebase Cloud Functions - Not used
❌ Node.js / TypeScript - Not used
❌ Python RAG Engine - Not used
❌ FAISS Vector Database - Not used
❌ Sentence Transformers - Not used
❌ OpenAI API - Not used
❌ Anthropic Claude - Not used
AI/ML Services:
- Vector Database: FAISS 1.7.4
- Embeddings: Sentence Transformers 2.2.2
- LLM APIs: OpenAI 4.20.1, Anthropic 0.6.3
- Processing: NumPy 1.21+, PyTorch 1.12+
Monitoring & Logging:
- Logging: Winston 3.8+
- Metrics: Prometheus Client
- Tracing: OpenTelemetry
- Error Tracking: Sentry
Actual Implementation:
✅ Flutter app calls Ollama directly via HTTP
✅ Firebase services handle auth, database, storage
✅ RAG logic implemented in Dart (keyword matching)
✅ Embeddings: Mock/hash-based in Dart
```
### Infrastructure Technologies
```yaml
Cloud Platform:
- Provider: Google Cloud Platform
- Services: Firebase, Cloud Functions, Cloud Storage
- Regions: us-central1, europe-west1
- CDN: Firebase Hosting
- Provider: Google Firebase (BaaS)
- Services: Firebase Auth, Firestore, Storage
- Hosting: Firebase Hosting (for web builds)
- Self-hosted: Ollama LLM server (89.114.196.110:11434)
Database:
- Primary: Cloud Firestore
- Cache: Redis (MemoryStore)
- Search: Elasticsearch (if needed)
- Backup: Automated daily backups
- Primary: Cloud Firestore (NoSQL)
- Cache: Hive (local), Memory cache
- Search: Not implemented (no Elasticsearch)
- Backup: Firebase automated backups
Security:
- TLS: 1.3
- Authentication: Firebase Auth
- Authorization: Custom RBAC
- Monitoring: Security Command Center
- TLS: HTTPS for all communications
- Authentication: Firebase Auth (email/password, Google)
- Authorization: Client-side role checks (student/teacher)
- Note: No admin role implemented
CI/CD:
- Pipeline: GitHub Actions
- Build: Cloud Build
- Deploy: Firebase CLI
- Testing: Automated test suites
- Pipeline: Manual builds
- Build: flutter build web/apk
- Deploy: Firebase CLI (manual)
- Testing: No automated tests implemented
```
---