Files

240403 3d3747d3a2 .md

2026-05-25 21:41:41 +01:00

47 KiB

Raw Permalink Blame History

Testing Strategy - AI Study Assistant

⚠️ ATUALIZADO: Este documento foi corrigido para refletir a arquitetura REAL.

Nota: O projeto é Flutter + Firebase BaaS. Não existe backend Node.js/Python para testar.

🧪 TESTING APPROACH

📋 OVERVIEW

This document outlines the testing strategy for the AI Study Assistant project (Flutter + Firebase BaaS). Testing focuses on Flutter unit/widget tests and Firebase security rules. There is no custom backend to test.

🎯 TESTING OBJECTIVES

Primary Goals:

Ensure Reliability: Maintain 99.5% uptime and error-free user experience
Validate Functionality: Verify all features work as expected
Performance Assurance: Ensure responsive UI and fast API responses
Security Validation: Verify data protection and access controls
User Experience: Test usability across different devices and platforms

Success Metrics:

Code Coverage: > 80% for business logic, > 60% for UI code
Test Pass Rate: > 95% for all automated tests
Performance: < 3s API response time, 60fps UI animations
Bug Detection: < 5 critical bugs in production

🏗️ TESTING ARCHITECTURE

Test Pyramid Structure:

    E2E Tests (5%)
   ─────────────────
  Integration Tests (15%)
 ─────────────────────────
Unit Tests (80%)
────────────────────────────────

Test Categories:

Unit Tests: Individual component testing
Widget Tests: UI component testing
Integration Tests: Component interaction testing
End-to-End Tests: Full user journey testing
Performance Tests: Load and stress testing
Security Tests: Vulnerability and penetration testing

📦 UNIT TESTING

1.1 Flutter Unit Tests

Test Structure:

test/
├── unit/
│   ├── core/
│   │   ├── services/
│   │   │   ├── test_storage_service.dart
│   │   │   ├── test_notification_service.dart
│   │   │   └── test_network_service.dart
│   │   ├── utils/
│   │   │   ├── test_validators.dart
│   │   │   ├── test_formatters.dart
│   │   │   └── test_extensions.dart
│   │   └── errors/
│   │       ├── test_exceptions.dart
│   │       └── test_failures.dart
│   ├── features/
│   │   ├── auth/
│   │   │   ├── domain/
│   │   │   │   ├── test_sign_in_usecase.dart
│   │   │   │   ├── test_sign_up_usecase.dart
│   │   │   │   └── test_user_repository.dart
│   │   │   ├── data/
│   │   │   │   ├── test_auth_remote_datasource.dart
│   │   │   │   └── test_auth_local_datasource.dart
│   │   │   └── presentation/
│   │   │       └── test_auth_provider.dart
│   │   ├── student/
│   │   │   ├── domain/
│   │   │   │   ├── test_learning_state_usecase.dart
│   │   │   │   ├── test_quiz_usecase.dart
│   │   │   │   └── test_chat_usecase.dart
│   │   │   └── presentation/
│   │   │       ├── test_student_provider.dart
│   │   │       └── test_chat_provider.dart
│   │   └── teacher/
│   │       ├── domain/
│   │       │   ├── test_content_upload_usecase.dart
│   │       │   └── test_quiz_creation_usecase.dart
│   │       └── presentation/
│   │           └── test_teacher_provider.dart
│   └── shared/
│       ├── domain/
│       │   ├── test_message_entity.dart
│       │   └── test_feedback_entity.dart
│       └── presentation/
│           └── test_shared_widgets.dart

Example Unit Test:

// test/unit/features/auth/domain/test_sign_in_usecase.dart
import 'package:flutter_test/flutter_test.dart';
import 'package:mockito/mockito.dart';
import 'package:dartz/dartz.dart';

import 'package:learn_it/features/auth/domain/entities/user.dart';
import 'package:learn_it/features/auth/domain/repositories/auth_repository.dart';
import 'package:learn_it/features/auth/domain/usecases/sign_in.dart';

class MockAuthRepository extends Mock implements AuthRepository {}

void main() {
  late SignIn usecase;
  late MockAuthRepository mockRepository;

  setUp(() {
    mockRepository = MockAuthRepository();
    usecase = SignIn(mockRepository);
  });

  const tEmail = 'test@example.com';
  const tPassword = 'password123';
  const tUser = User(
    id: '1',
    email: tEmail,
    role: 'student',
    profile: UserProfile(name: 'Test User'),
  );

  test('should sign in user with valid credentials', () async {
    // Arrange
    when(mockRepository.signIn(any(), any()))
        .thenAnswer((_) async => Right(tUser));

    // Act
    final result = await usecase(const SignInParams(
      email: tEmail,
      password: tPassword,
    ));

    // Assert
    expect(result, Right(tUser));
    verify(mockRepository.signIn(tEmail, tPassword)).called(1);
  });

  test('should return failure when credentials are invalid', () async {
    // Arrange
    when(mockRepository.signIn(any(), any()))
        .thenAnswer((_) async => Left(AuthFailure.invalidCredentials()));

    // Act
    final result = await usecase(const SignInParams(
      email: 'invalid@email.com',
      password: 'wrong',
    ));

    // Assert
    expect(result, Left(AuthFailure.invalidCredentials()));
    verify(mockRepository.signIn(any(), any())).called(1);
  });

  test('should handle network errors', () async {
    // Arrange
    when(mockRepository.signIn(any(), any()))
        .thenThrow(Exception('Network error'));

    // Act
    final result = await usecase(const SignInParams(
      email: tEmail,
      password: tPassword,
    ));

    // Assert
    expect(result, Left(AuthFailure.networkError()));
    verify(mockRepository.signIn(tEmail, tPassword)).called(1);
  });
}

1.2 Backend Unit Tests

Test Structure:

functions/test/
├── unit/
│   ├── services/
│   │   ├── test_auth_service.py
│   │   ├── test_rag_service.py
│   │   ├── test_llm_service.py
│   │   └── test_content_service.py
│   ├── core/
│   │   ├── test_vector_store.py
│   │   ├── test_embeddings.py
│   │   └── test_retriever.py
│   ├── preprocessing/
│   │   ├── test_text_processor.py
│   │   └── test_chunker.py
│   └── utils/
│       ├── test_validators.py
│       └── test_helpers.py

Example Backend Unit Test:

# functions/test/unit/services/test_rag_service.py
import pytest
from unittest.mock import Mock, patch
from src.services.rag_service import RAGService
from src.models.query import Query
from src.models.chunk import Chunk

class TestRAGService:
    def setup_method(self):
        self.mock_embedding_service = Mock()
        self.mock_vector_store = Mock()
        self.mock_llm_service = Mock()
        self.mock_content_service = Mock()
        
        self.rag_service = RAGService(
            embedding_service=self.mock_embedding_service,
            vector_store=self.mock_vector_store,
            llm_service=self.mock_llm_service,
            content_service=self.mock_content_service
        )

    @pytest.mark.asyncio
    async def test_process_student_query_success(self):
        # Arrange
        query_text = "What is a derivative?"
        student_id = "student123"
        
        query = Query(
            text=query_text,
            student_id=student_id,
            mode="EXPLANATION"
        )
        
        learning_state = {
            "adaptiveDifficulty": {"currentLevel": 2},
            "schoolId": "school123"
        }
        
        retrieved_chunks = [
            Chunk(
                id="chunk1",
                text="A derivative measures the rate of change...",
                score=0.9
            )
        ]
        
        llm_response = {
            "text": "A derivative is a concept that measures how a function changes...",
            "tokens": 150,
            "model": "claude-3-5-sonnet"
        }
        
        self.mock_content_service.get_learning_state.return_value = learning_state
        self.mock_vector_store.search.return_value = [("chunk1", 0.9)]
        self.mock_llm_service.generate_response.return_value = llm_response
        
        # Act
        result = await self.rag_service.process_student_query(
            student_id, query_text, "EXPLANATION"
        )
        
        # Assert
        assert result["success"] is True
        assert "response" in result
        assert result["response"] == llm_response["text"]
        assert result["retrieved_chunks"] == [("chunk1", 0.9)]
        
        self.mock_vector_store.search.assert_called_once()
        self.mock_llm_service.generate_response.assert_called_once()

    @pytest.mark.asyncio
    async def test_process_student_query_no_context(self):
        # Arrange
        query_text = "What is quantum physics?"
        student_id = "student123"
        
        self.mock_vector_store.search.return_value = []
        
        # Act
        result = await self.rag_service.process_student_query(
            student_id, query_text, "EXPLANATION"
        )
        
        # Assert
        assert result["success"] is True
        assert result["no_context"] is True
        assert "don't have relevant information" in result["response"]

    @pytest.mark.asyncio
    async def test_process_student_query_llm_error(self):
        # Arrange
        query_text = "What is a derivative?"
        student_id = "student123"
        
        self.mock_vector_store.search.return_value = [("chunk1", 0.9)]
        self.mock_llm_service.generate_response.side_effect = Exception("LLM Error")
        
        # Act & Assert
        with pytest.raises(Exception, match="LLM Error"):
            await self.rag_service.process_student_query(
                student_id, query_text, "EXPLANATION"
            )

Test Structure:

test/widget/
├── features/
│   ├── auth/
│   │   ├── test_login_screen.dart
│   │   ├── test_signup_screen.dart
│   │   ├── test_auth_form.dart
│   │   └── test_social_login_button.dart
│   ├── student/
│   │   ├── test_dashboard_screen.dart
│   │   ├── test_chat_screen.dart
│   │   ├── test_quiz_screen.dart
│   │   └── test_progress_screen.dart
│   ├── teacher/
│   │   ├── test_teacher_dashboard.dart
│   │   ├── test_content_upload.dart
│   │   └── test_quiz_creation.dart
│   └── shared/
│       ├── test_primary_button.dart
│       ├── test_custom_text_field.dart
│       ├── test_standard_card.dart
│       └── test_chat_bubble.dart

// test/widget/features/auth/test_login_screen.dart
import 'package:flutter/material.dart';
import 'package:flutter_test/flutter_test.dart';
import 'package:flutter_riverpod/flutter_riverpod.dart';
import 'package:mockito/mockito.dart';
import 'package:mockito/annotations.dart';

import 'package:teachit/features/auth/presentation/screens/login_screen.dart';
import 'package:learn_it/features/auth/presentation/providers/auth_provider.dart';

import 'login_screen_test.mocks.dart';

@GenerateMocks([AuthProvider])
void main() {
  group('LoginScreen Widget Tests', () {
    late MockAuthProvider mockAuthProvider;

    setUp(() {
      mockAuthProvider = MockAuthProvider();
    });

    Widget createWidgetUnderTest() {
      return ProviderScope(
        overrides: [
          authProvider.overrideWithValue(mockAuthProvider),
        ],
        child: MaterialApp(
          home: LoginScreen(),
        ),
      );
    }

    testWidgets('should display login form elements', (WidgetTester tester) async {
      // Arrange
      await tester.pumpWidget(createWidgetUnderTest());

      // Assert
      expect(find.byType(TextField), findsNWidgets(2)); // Email and password
      expect(find.byType(ElevatedButton), findsOneWidget);
      expect(find.text('Sign In'), findsOneWidget);
      expect(find.text('Forgot Password?'), findsOneWidget);
      expect(find.text("Don't have an account? Sign Up"), findsOneWidget);
    });

    testWidgets('should enable sign in button when form is valid', (WidgetTester tester) async {
      // Arrange
      await tester.pumpWidget(createWidgetUnderTest());

      // Act
      await tester.enterText(find.byKey(Key('email_field')), 'test@example.com');
      await tester.enterText(find.byKey(Key('password_field')), 'password123');
      await tester.pump();

      // Assert
      final button = tester.widget<ElevatedButton>(find.byType(ElevatedButton));
      expect(button.onPressed != null, isTrue);
    });

    testWidgets('should disable sign in button when form is invalid', (WidgetTester tester) async {
      // Arrange
      await tester.pumpWidget(createWidgetUnderTest());

      // Act
      await tester.enterText(find.byKey(Key('email_field')), 'invalid-email');
      await tester.pump();

      // Assert
      final button = tester.widget<ElevatedButton>(find.byType(ElevatedButton));
      expect(button.onPressed, isNull);
    });

    testWidgets('should call signIn when sign in button is pressed', (WidgetTester tester) async {
      // Arrange
      await tester.pumpWidget(createWidgetUnderTest());
      
      when(mockAuthProvider.signIn(any(), any()))
          .thenAnswer((_) async {});

      // Act
      await tester.enterText(find.byKey(Key('email_field')), 'test@example.com');
      await tester.enterText(find.byKey(Key('password_field')), 'password123');
      await tester.pump();
      await tester.tap(find.byType(ElevatedButton));
      await tester.pump();

      // Assert
      verify(mockAuthProvider.signIn('test@example.com', 'password123')).called(1);
    });

    testWidgets('should show error message when sign in fails', (WidgetTester tester) async {
      // Arrange
      await tester.pumpWidget(createWidgetUnderTest());
      
      when(mockAuthProvider.signIn(any(), any()))
          .thenThrow(Exception('Invalid credentials'));
      
      // Act
      await tester.enterText(find.byKey(Key('email_field')), 'test@example.com');
      await tester.enterText(find.byKey(Key('password_field')), 'wrong');
      await tester.pump();
      await tester.tap(find.byType(ElevatedButton));
      await tester.pump();

      // Assert
      expect(find.text('Login failed: Exception: Invalid credentials'), findsOneWidget);
    });

    testWidgets('should navigate to sign up screen when sign up is pressed', (WidgetTester tester) async {
      // Arrange
      await tester.pumpWidget(createWidgetUnderTest());

      // Act
      await tester.tap(find.text('Sign Up'));
      await tester.pumpAndSettle();

      // Assert
      expect(find.byType(SignUpScreen), findsOneWidget);
    });

    testWidgets('should show loading indicator during sign in', (WidgetTester tester) async {
      // Arrange
      await tester.pumpWidget(createWidgetUnderTest());
      
      when(mockAuthProvider.signIn(any(), any()))
          .thenAnswer((_) async {
        await Future.delayed(Duration(seconds: 1));
      });
      
      // Act
      await tester.enterText(find.byKey(Key('email_field')), 'test@example.com');
      await tester.enterText(find.byKey(Key('password_field')), 'password123');
      await tester.pump();
      await tester.tap(find.byType(ElevatedButton));
      await tester.pump();

      // Assert
      expect(find.byType(CircularProgressIndicator), findsOneWidget);
      expect(find.byType(ElevatedButton), findsNothing);
    });
  });
}

🔗 INTEGRATION TESTING

3.1 Flutter Integration Tests

Test Structure:

test/integration/
├── auth_flow_test.dart
├── student_dashboard_test.dart
├── teacher_dashboard_test.dart
├── chat_functionality_test.dart
├── quiz_flow_test.dart
├── content_upload_test.dart
└── analytics_test.dart

Example Integration Test:

// test/integration/auth_flow_test.dart
import 'package:flutter/material.dart';
import 'package:flutter_test/flutter_test.dart';
import 'package:flutter_riverpod/flutter_riverpod.dart';
import 'package:firebase_core/firebase_core.dart';
import 'package:cloud_firestore/cloud_firestore.dart';
import 'package:firebase_auth/firebase_auth.dart';

import 'package:learn_it/main.dart' as app;
import 'package:teachit/features/auth/presentation/screens/login_screen.dart';
import 'package:learn_it/features/student/presentation/screens/student_dashboard_screen.dart';

void main() {
  group('Authentication Flow Integration Tests', () {
    setUpAll(() async {
      // Initialize Firebase for testing
      await Firebase.initializeApp();
      FirebaseFirestore.instance.useFirestoreEmulator('localhost', 8080);
      FirebaseAuth.instance.useAuthEmulator('localhost', 9099);
    });

    testWidgets('complete authentication flow', (WidgetTester tester) async {
      // Arrange
      await tester.pumpWidget(ProviderScope(child: app.MyApp()));

      // Step 1: Navigate to login screen
      expect(find.byType(LoginScreen), findsOneWidget);

      // Step 2: Fill in login form
      await tester.enterText(find.byKey(Key('email_field')), 'student@test.com');
      await tester.enterText(find.byKey(Key('password_field')), 'password123');
      await tester.pump();

      // Step 3: Sign in
      await tester.tap(find.byType(ElevatedButton));
      await tester.pumpAndSettle(Duration(seconds: 3));

      // Assert: Should be on student dashboard
      expect(find.byType(StudentDashboardScreen), findsOneWidget);
      
      // Assert: User should be authenticated
      final currentUser = FirebaseAuth.instance.currentUser;
      expect(currentUser, isNotNull);
      expect(currentUser!.email, equals('student@test.com'));
    });

    testWidgets('should handle authentication error gracefully', (WidgetTester tester) async {
      // Arrange
      await tester.pumpWidget(ProviderScope(child: app.MyApp()));

      // Step 1: Try to sign in with invalid credentials
      await tester.enterText(find.byKey(Key('email_field')), 'invalid@test.com');
      await tester.enterText(find.byKey(Key('password_field')), 'wrong');
      await tester.pump();

      // Step 2: Attempt sign in
      await tester.tap(find.byType(ElevatedButton));
      await tester.pump();

      // Assert: Should show error message
      expect(find.text('Login failed'), findsOneWidget);
      
      // Assert: Should remain on login screen
      expect(find.byType(LoginScreen), findsOneWidget);
      expect(find.byType(StudentDashboardScreen), findsNothing);
    });
  });
}

3.2 Backend Integration Tests

Test Structure:

functions/test/integration/
├── test_auth_flow.py
├── test_content_processing.py
├── test_rag_pipeline.py
├── test_quiz_system.py
├── test_analytics.py
└── test_firebase_integration.py

Example Integration Test:

# functions/test/integration/test_rag_pipeline.py
import pytest
import asyncio
from firebase_admin import firestore, auth
from src.services.rag_service import RAGService
from src.services.content_service import ContentService
from src.core.embeddings import EmbeddingService
from src.core.vector_store import VectorStore

@pytest.mark.integration
class TestRAGPipelineIntegration:
    
    @pytest.fixture
    async def rag_service(self):
        """Initialize RAG service with real Firebase"""
        embedding_service = EmbeddingService()
        vector_store = VectorStore()
        content_service = ContentService()
        
        return RAGService(
            embedding_service=embedding_service,
            vector_store=vector_store,
            content_service=content_service
        )
    
    @pytest.fixture
    async def test_student(self):
        """Create test student"""
        user = auth.create_user(
            email='test@student.com',
            password='password123'
        )
        
        # Create learning state
        db = firestore.client()
        await db.collection('learningStates').document(user.uid).set({
            'studentId': user.uid,
            'schoolId': 'test-school',
            'adaptiveDifficulty': {'currentLevel': 2},
            'conceptStates': {},
            'spacedRepetition': {'nextReviewDue': []},
            'metadata': {'totalInteractions': 0}
        })
        
        return user.uid
    
    @pytest.mark.asyncio
    async def test_end_to_end_query_processing(self, rag_service, test_student):
        """Test complete query processing pipeline"""
        
        # Step 1: Upload test content
        content_text = """
        [CONCEPT_START: Derivatives]
        A derivative measures the rate of change of a function with respect to a variable.
        For example, if f(x) = x², then f'(x) = 2x.
        [CONCEPT_END]
        """
        
        upload_result = await rag_service.content_service.process_uploaded_file(
            teacher_id='teacher123',
            school_id='test-school',
            file=content_text.encode(),
            fileName='derivatives.txt',
            mimeType='text/plain',
            metadata={
                'subject': 'Mathematics',
                'concept': 'Derivatives',
                'difficulty': 0.5
            }
        )
        
        assert upload_result['success'] is True
        assert upload_result['chunk_count'] > 0
        
        # Step 2: Process student query
        query_result = await rag_service.process_student_query(
            student_id=test_student,
            query="What is a derivative?",
            mode="EXPLANATION"
        )
        
        # Step 3: Verify results
        assert query_result['success'] is True
        assert 'response' in query_result
        assert len(query_result['response']) > 0
        assert query_result['retrieved_chunks'] is not None
        assert len(query_result['retrieved_chunks']) > 0
        
        # Step 4: Verify response quality
        response = query_result['response']
        assert 'derivative' in response.lower()
        assert 'rate of change' in response.lower()
        
        # Step 5: Verify interaction logged
        db = firestore.client()
        interactions = db.collection('interactions').where(
            'studentId', '==', test_student
        ).get()
        
        assert len(interactions) > 0
        interaction = interactions[0].to_dict()
        assert interaction['query'] == "What is a derivative?"
        assert interaction['mode'] == "EXPLANATION"
        assert interaction['response'] == response

    @pytest.mark.asyncio
    async def test_content_upload_and_retrieval(self, rag_service):
        """Test content upload and retrieval pipeline"""
        
        # Step 1: Upload content
        content_text = """
        The chain rule is a formula for computing the derivative of the composition of two or more functions.
        If y = f(u) and u = g(x), then dy/dx = dy/du * du/dx.
        """
        
        upload_result = await rag_service.content_service.process_uploaded_file(
            teacher_id='teacher123',
            school_id='test-school',
            file=content_text.encode(),
            fileName='chain_rule.txt',
            mimeType='text/plain',
            metadata={
                'subject': 'Mathematics',
                'concept': 'Chain Rule',
                'difficulty': 0.7
            }
        )
        
        assert upload_result['success'] is True
        
        # Step 2: Query for uploaded content
        query_result = await rag_service.process_student_query(
            student_id='test-student',
            query="How do I use the chain rule?",
            mode="TUTOR"
        )
        
        # Step 3: Verify retrieval worked
        assert query_result['success'] is True
        assert 'chain rule' in query_result['response'].lower()
        assert len(query_result['retrieved_chunks']) > 0
        
        # Step 4: Verify chunk metadata
        for chunk_id, score in query_result['retrieved_chunks']:
            chunk = await rag_service.content_service.get_chunk(chunk_id)
            assert chunk['concept'] == 'Chain Rule'
            assert chunk['subject'] == 'Mathematics'

🎭 END-TO-END TESTING

4.1 E2E Test Scenarios

Test Structure:

integration_test/
├── app_test.dart
├── auth_flow_test.dart
├── student_learning_journey_test.dart
├── teacher_content_management_test.dart
├── quiz_creation_and_taking_test.dart
├── cross_platform_test.dart
└── performance_test.dart

Example E2E Test:

// integration_test/student_learning_journey_test.dart
import 'package:flutter/material.dart';
import 'package:flutter_test/flutter_test.dart';
import 'package:integration_test/integration_test.dart';

import 'package:learn_it/main.dart' as app;

void main() {
  IntegrationTestWidgetsFlutterBinding.ensureInitialized();

  group('Student Learning Journey E2E Tests', () {
    late IntegrationTestWidgetsFlutterBinding binding;

    setUp(() {
      binding = IntegrationTestWidgetsFlutterBinding.ensureInitialized();
    });

    testWidgets('complete student learning journey', (WidgetTester tester) async {
      // Step 1: Launch app
      app.main();
      await tester.pumpAndSettle();

      // Step 2: Sign in
      await tester.enterText(find.byKey(Key('email_field')), 'student@test.com');
      await tester.enterText(find.byKey(Key('password_field')), 'password123');
      await tester.tap(find.byType(ElevatedButton));
      await tester.pumpAndSettle(Duration(seconds: 3));

      // Step 3: Verify dashboard
      expect(find.text('My Learning'), findsOneWidget);
      expect(find.byType(CircularProgressIndicator), findsOneWidget); // Loading progress

      // Step 4: Ask tutor a question
      await tester.tap(find.text('Ask Tutor'));
      await tester.pumpAndSettle();

      await tester.enterText(find.byType(TextField), 'What is a derivative?');
      await tester.tap(find.byIcon(Icons.send));
      await tester.pumpAndSettle(Duration(seconds: 5));

      // Step 5: Verify response
      expect(find.textContaining('derivative'), findsOneWidget);
      expect(find.textContaining('rate of change'), findsOneWidget);

      // Step 6: Provide feedback
      await tester.tap(find.byIcon(Icons.thumb_up));
      await tester.pumpAndSettle();

      // Step 7: Take a quiz
      await tester.tap(find.text('Dashboard'));
      await tester.pumpAndSettle();
      await tester.tap(find.text('Take Quiz'));
      await tester.pumpAndSettle();

      // Answer quiz questions
      await tester.tap(find.text('f(x) = x²'));
      await tester.pumpAndSettle();
      await tester.tap(find.text('2x'));
      await tester.pumpAndSettle();
      await tester.tap(find.text('Submit'));
      await tester.pumpAndSettle(Duration(seconds: 2));

      // Step 8: Verify quiz results
      expect(find.textContaining('Quiz Complete'), findsOneWidget);
      expect(find.textContaining('Score:'), findsOneWidget);

      // Step 9: Check progress
      await tester.tap(find.text('Progress'));
      await tester.pumpAndSettle();

      // Verify progress tracking
      expect(find.byType(LinearProgressIndicator), findsAtLeastNWidgets(1));
      expect(find.textContaining('Mastery'), findsOneWidget);

      // Step 10: Sign out
      await tester.tap(find.byIcon(Icons.logout));
      await tester.pumpAndSettle();

      // Verify back to login screen
      expect(find.byType(LoginScreen), findsOneWidget);
    });

    testWidgets('should handle network disconnection gracefully', (WidgetTester tester) async {
      // Step 1: Launch app and sign in
      app.main();
      await tester.pumpAndSettle();

      await tester.enterText(find.byKey(Key('email_field')), 'student@test.com');
      await tester.enterText(find.byKey(Key('password_field')), 'password123');
      await tester.tap(find.byType(ElevatedButton));
      await tester.pumpAndSettle(Duration(seconds: 3));

      // Step 2: Simulate network disconnection
      binding.binding.defaultBinaryMessenger.handlePlatformMessage(
        'flutter/network',
        null,
        (data) {},
      );

      // Step 3: Try to ask a question
      await tester.tap(find.text('Ask Tutor'));
      await tester.pumpAndSettle();

      await tester.enterText(find.byType(TextField), 'Test question');
      await tester.tap(find.byIcon(Icons.send));
      await tester.pumpAndSettle(Duration(seconds: 3));

      // Step 4: Verify offline handling
      expect(find.text('No internet connection'), findsOneWidget);
      expect(find.text('Retry'), findsOneWidget);

      // Step 5: Restore connection and retry
      binding.binding.defaultBinaryMessenger.handlePlatformMessage(
        'flutter/network',
        null,
        (data) {},
      );

      await tester.tap(find.text('Retry'));
      await tester.pumpAndSettle(Duration(seconds: 5));

      // Verify request succeeded
      expect(find.textContaining('Test question'), findsOneWidget);
    });
  });
}

⚡ PERFORMANCE TESTING

5.1 Load Testing

Load Test Scenarios:

Concurrent Users: 100+ simultaneous users
API Response Times: < 500ms for 95% of requests
Database Performance: < 100ms query times
Memory Usage: < 512MB per user session
CPU Usage: < 70% under normal load

Load Test Implementation:

# functions/test/performance/load_test.py
import asyncio
import aiohttp
import time
from concurrent.futures import ThreadPoolExecutor
import statistics

class LoadTester:
    def __init__(self, base_url: str):
        self.base_url = base_url
        self.results = []

    async def simulate_user_session(self, user_id: int):
        """Simulate a complete user session"""
        start_time = time.time()
        session_times = []
        
        async with aiohttp.ClientSession() as session:
            try:
                # Sign in
                signin_start = time.time()
                async with session.post(
                    f"{self.base_url}/auth/signin",
                    json={
                        "email": f"user{user_id}@test.com",
                        "password": "password123"
                    }
                ) as response:
                    await response.json()
                session_times.append(time.time() - signin_start)

                # Ask tutor question
                question_start = time.time()
                async with session.post(
                    f"{self.base_url}/tutor/ask",
                    json={
                        "query": "What is a derivative?",
                        "mode": "EXPLANATION"
                    }
                ) as response:
                    await response.json()
                session_times.append(time.time() - question_start)

                # Take quiz
                quiz_start = time.time()
                async with session.post(
                    f"{self.base_url}/quiz/submit",
                    json={
                        "quizId": "test-quiz",
                        "answers": {"q1": "A", "q2": "B"}
                    }
                ) as response:
                    await response.json()
                session_times.append(time.time() - quiz_start)

            except Exception as e:
                print(f"User {user_id} session failed: {e}")
                return None

        total_time = time.time() - start_time
        return {
            "user_id": user_id,
            "total_time": total_time,
            "session_times": session_times,
            "avg_response_time": statistics.mean(session_times) if session_times else 0
        }

    async def run_load_test(self, concurrent_users: int, duration_seconds: int):
        """Run load test with specified concurrent users"""
        print(f"Starting load test: {concurrent_users} concurrent users")
        
        start_time = time.time()
        tasks = []
        
        # Create concurrent user sessions
        for i in range(concurrent_users):
            task = asyncio.create_task(self.simulate_user_session(i))
            tasks.append(task)
        
        # Wait for all tasks to complete or timeout
        try:
            results = await asyncio.wait_for(
                asyncio.gather(*tasks, return_exceptions=True),
                timeout=duration_seconds
            )
        except asyncio.TimeoutError:
            print("Load test timed out")
            return

        # Analyze results
        successful_sessions = [r for r in results if isinstance(r, dict)]
        failed_sessions = [r for r in results if isinstance(r, Exception)]

        if successful_sessions:
            avg_session_time = statistics.mean([r["total_time"] for r in successful_sessions])
            avg_response_time = statistics.mean([r["avg_response_time"] for r in successful_sessions])
            
            print(f"Load Test Results:")
            print(f"  Successful sessions: {len(successful_sessions)}/{concurrent_users}")
            print(f"  Failed sessions: {len(failed_sessions)}")
            print(f"  Average session time: {avg_session_time:.2f}s")
            print(f"  Average response time: {avg_response_time:.2f}s")
            print(f"  Success rate: {len(successful_sessions)/concurrent_users*100:.1f}%")
        else:
            print("No successful sessions")

# Usage
async def main():
    tester = LoadTester("http://localhost:5001")
    await tester.run_load_test(concurrent_users=100, duration_seconds=60)

if __name__ == "__main__":
    asyncio.run(main())

5.2 Performance Monitoring

Flutter Performance Tests:

// test/performance/app_performance_test.dart
import 'package:flutter_test/flutter_test.dart';
import 'package:flutter_driver/flutter_driver.dart';
import 'package:test/test.dart' as test;

void main() {
  group('App Performance Tests', () {
    late FlutterDriver driver;

    setUpAll(() async {
      driver = await FlutterDriver.connect();
    });

    tearDownAll(() async {
      await driver?.close();
    });

    test('should render dashboard within performance limits', () async {
      // Navigate to dashboard
      await driver.tap(find.byValueKey('dashboard_tab'));
      await driver.waitFor(find.byValueKey('dashboard_loaded'));

      // Measure rendering performance
      final timeline = await driver.traceAction('dashboard_render');
      
      // Assert performance metrics
      expect(timeline.frames.length, greaterThan(0));
      expect(timeline.averageFrameTime, lessThan(16.67)); // 60fps
      expect(timeline.worstFrameTime, lessThan(33.33)); // 30fps minimum
    });

    test('should handle chat interactions smoothly', () async {
      // Navigate to chat
      await driver.tap(find.byValueKey('chat_tab'));
      await driver.waitFor(find.byValueKey('chat_loaded'));

      // Measure typing performance
      final timeline = await driver.traceAction('chat_typing');
      
      await driver.tap(find.byValueKey('chat_input'));
      await driver.enterText('What is a derivative?');
      await driver.tap(find.byValueKey('send_button'));
      
      // Assert smooth interaction
      expect(timeline.frames.length, greaterThan(0));
      expect(timeline.averageFrameTime, lessThan(16.67));
    });

    test('should maintain memory usage within limits', () async {
      // Monitor memory usage during session
      final initialMemory = await driver.getMemoryUsage();
      
      // Perform various actions
      await driver.tap(find.byValueKey('dashboard_tab'));
      await driver.tap(find.byValueKey('chat_tab'));
      await driver.tap(find.byValueKey('quiz_tab'));
      
      // Check memory usage
      final finalMemory = await driver.getMemoryUsage();
      final memoryIncrease = finalMemory - initialMemory;
      
      // Assert memory usage is reasonable (< 100MB increase)
      expect(memoryIncrease, lessThan(100 * 1024 * 1024));
    });
  });
}

🔒 SECURITY TESTING

6.1 Security Test Scenarios

Test Categories:

Authentication Security: Password strength, session management
Data Protection: Encryption, access controls
API Security: Rate limiting, input validation
Cross-Site Scripting: XSS prevention
SQL Injection: Query parameter validation
Authorization: Role-based access control

Security Test Implementation:

# functions/test/security/security_test.py
import pytest
import requests
import json
from test.integration.test_base import IntegrationTestBase

class SecurityTests(IntegrationTestBase):
    
    def test_sql_injection_prevention(self):
        """Test SQL injection prevention"""
        malicious_inputs = [
            "'; DROP TABLE users; --",
            "' OR '1'='1",
            "'; DELETE FROM learningStates; --",
            "admin'--"
        ]
        
        for malicious_input in malicious_inputs:
            response = self.client.post('/auth/signin', json={
                'email': malicious_input,
                'password': 'password123'
            })
            
            # Should not authenticate with malicious input
            assert response.status_code in [400, 401, 403]
    
    def test_xss_prevention(self):
        """Test XSS prevention in content"""
        xss_payloads = [
            "<script>alert('xss')</script>",
            "javascript:alert('xss')",
            "<img src=x onerror=alert('xss')>",
            "'\"><script>alert('xss')</script>"
        ]
        
        for payload in xss_payloads:
            # Test content upload
            response = self.client.post('/content/upload', json={
                'text': payload,
                'concept': 'Test Concept',
                'subject': 'Test Subject'
            })
            
            if response.status_code == 200:
                # Content should be sanitized
                content = response.json()['content']
                assert '<script>' not in content
                assert 'javascript:' not in content
    
    def test_rate_limiting(self):
        """Test API rate limiting"""
        # Make rapid requests
        responses = []
        for i in range(150):  # Above rate limit
            response = self.client.post('/tutor/ask', json={
                'query': f'Test question {i}',
                'mode': 'EXPLANATION'
            })
            responses.append(response.status_code)
        
        # Should be rate limited after certain number of requests
        assert 429 in responses  # Too Many Requests
    
    def test_authorization_bypass(self):
        """Test authorization bypass attempts"""
        # Test accessing teacher endpoints as student
        student_token = self.get_auth_token('student@test.com', 'password123')
        
        protected_endpoints = [
            '/content/upload',
            '/quiz/create',
            '/analytics/class'
        ]
        
        for endpoint in protected_endpoints:
            response = self.client.post(
                endpoint,
                json={'test': 'data'},
                headers={'Authorization': f'Bearer {student_token}'}
            )
            
            # Should be forbidden for student
            assert response.status_code in [401, 403]
    
    def test_sensitive_data_exposure(self):
        """Test for sensitive data exposure"""
        # Test API responses for sensitive information
        endpoints = [
            '/auth/profile',
            '/learning/state',
            '/quiz/attempts'
        ]
        
        for endpoint in endpoints:
            response = self.client.get(
                endpoint,
                headers={'Authorization': f'Bearer {self.auth_token}'}
            )
            
            if response.status_code == 200:
                data = response.json()
                
                # Check for sensitive data exposure
                sensitive_fields = ['password', 'token', 'secret', 'key']
                for field in sensitive_fields:
                    assert field not in str(data).lower()
    
    def test_file_upload_security(self):
        """Test file upload security"""
        malicious_files = [
            ('malicious.exe', b'fake executable content'),
            ('script.php', b'<?php echo "hack"; ?>'),
            ('large_file.txt', b'x' * (50 * 1024 * 1024))  # 50MB
        ]
        
        for filename, content in malicious_files:
            response = self.client.post(
                '/content/upload_file',
                files={'file': (filename, content)}
            )
            
            # Should reject malicious or oversized files
            assert response.status_code in [400, 413]

📊 TEST AUTOMATION

7.1 CI/CD Pipeline Tests

GitHub Actions Configuration:

# .github/workflows/test.yml
name: Test Suite

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

jobs:
  flutter-tests:
    runs-on: ubuntu-latest
    
    steps:
      - uses: actions/checkout@v3
      
      - name: Setup Flutter
        uses: subosito/flutter-action@v2
        with:
          flutter-version: '3.41.0'
      
      - name: Install dependencies
        run: flutter pub get
      
      - name: Run unit tests
        run: flutter test --coverage
      
      - name: Run widget tests
        run: flutter test test/widget/
      
      - name: Upload coverage
        uses: codecov/codecov-action@v3
        with:
          file: coverage/lcov.info

  backend-tests:
    runs-on: ubuntu-latest
    
    steps:
      - uses: actions/checkout@v3
      
      - name: Setup Node.js
        uses: actions/setup-node@v3
        with:
          node-version: '18'
      
      - name: Install dependencies
        run: |
          cd functions
          npm install
      
      - name: Run unit tests
        run: |
          cd functions
          npm test
      
      - name: Run integration tests
        run: |
          cd functions
          npm run test:integration

  e2e-tests:
    runs-on: ubuntu-latest
    
    steps:
      - uses: actions/checkout@v3
      
      - name: Setup Flutter
        uses: subosito/flutter-action@v2
        with:
          flutter-version: '3.41.0'
      
      - name: Start Firebase emulators
        run: |
          firebase emulators:start --only firestore,auth,functions &
          sleep 30
      
      - name: Run E2E tests
        run: flutter test integration_test/

7.2 Test Reporting

Test Coverage Reporting:

// test/test_utils/coverage_reporter.dart
import 'dart:io';
import 'package:test_reporter/test_reporter.dart';

class CoverageReporter {
  static Future<void> generateReport() async {
    final result = await Process.run('flutter', ['test', '--coverage']);
    
    if (result.exitCode == 0) {
      // Generate HTML coverage report
      await Process.run('genhtml', ['coverage/lcov.info', '-o', 'coverage/html']);
      
      // Generate JSON report
      final coverageData = await _parseCoverageFile('coverage/lcov.info');
      await _generateJsonReport(coverageData);
      
      print('Coverage report generated successfully!');
    } else {
      print('Failed to generate coverage report');
    }
  }
  
  static Future<Map<String, dynamic>> _parseCoverageFile(String filePath) async {
    final file = File(filePath);
    final content = await file.readAsString();
    
    // Parse LCov format
    final lines = content.split('\n');
    final coverageData = <String, dynamic>{};
    
    for (final line in lines) {
      if (line.startsWith('SF:')) {
        coverageData['source_file'] = line.substring(3);
      } else if (line.startsWith('LH:')) {
        // Line hit count
        final parts = line.split(':');
        coverageData['line_hits'] = int.parse(parts[1]);
      }
    }
    
    return coverageData;
  }
  
  static Future<void> _generateJsonReport(Map<String, dynamic> coverageData) async {
    final report = {
      'timestamp': DateTime.now().toIso8601String(),
      'coverage': coverageData,
      'summary': {
        'total_lines': coverageData['total_lines'] ?? 0,
        'covered_lines': coverageData['covered_lines'] ?? 0,
        'coverage_percentage': coverageData['coverage_percentage'] ?? 0.0,
      }
    };
    
    final file = File('coverage/coverage_report.json');
    await file.writeAsString(jsonEncode(report));
  }
}

📋 TESTING CHECKLIST

Pre-Release Testing:

All unit tests passing (>80% coverage)
All widget tests passing
All integration tests passing
E2E tests covering critical user journeys
Performance tests meeting benchmarks
Security tests passing
Accessibility tests passing
Cross-platform compatibility verified
Load testing completed
Documentation updated

Continuous Testing:

Automated test suite in CI/CD
Code coverage reporting
Performance regression detection
Security vulnerability scanning
Dependency vulnerability checks
Test environment provisioning
Test data management
Result reporting and alerts

Production Monitoring:

Real-time error tracking
Performance monitoring
User experience metrics
A/B testing framework
Canary deployment testing
Rollback procedures
Incident response plan

🛠️ TESTING TOOLS & FRAMEWORKS

Flutter Testing:

flutter_test: Built-in testing framework
integration_test: E2E testing
flutter_driver: Performance testing
mockito: Mocking framework
golden_tests: UI regression testing

Backend Testing:

pytest: Python testing framework
pytest-asyncio: Async testing
mock: Python mocking
requests-mock: HTTP mocking
testcontainers: Container testing

Performance Testing:

locust: Load testing
k6: Performance testing
gatling: Load testing
JMeter: Performance testing

Security Testing:

OWASP ZAP: Security scanning
Burp Suite: Security testing
sqlmap: SQL injection testing
nmap: Network scanning

📚 TESTING BEST PRACTICES

General Principles:

Test Early: Write tests alongside code
Test Often: Run tests frequently
Test Thoroughly: Cover edge cases and error conditions
Test Independently: Tests should not depend on each other
Test Readably: Clear test names and documentation

Test Organization:

Arrange-Act-Assert: Clear test structure
Descriptive Names: Test names should describe what they test
Single Responsibility: Each test should test one thing
Reusable Fixtures: Common test setup and teardown
Test Data Management: Consistent test data approach

Test Maintenance:

Regular Updates: Keep tests updated with code changes
Refactoring: Refactor tests when refactoring code
Documentation: Document complex test scenarios
Performance: Optimize slow tests
Monitoring: Monitor test effectiveness

Last Updated: 2026-05-06 Version: 1.0.0 Testing Lead: QA Development Team

47 KiB Raw Permalink Blame History

Testing Strategy - AI Study Assistant

🧪 TESTING APPROACH

📋 OVERVIEW

🎯 TESTING OBJECTIVES

Primary Goals:

Success Metrics:

🏗️ TESTING ARCHITECTURE

Test Pyramid Structure:

Test Categories:

📦 UNIT TESTING

1.1 Flutter Unit Tests

Test Structure:

Example Unit Test:

1.2 Backend Unit Tests

Test Structure:

Example Backend Unit Test:

🎨 WIDGET TESTING

2.1 Flutter Widget Tests

Test Structure:

Example Widget Test:

🔗 INTEGRATION TESTING

3.1 Flutter Integration Tests

Test Structure:

Example Integration Test:

3.2 Backend Integration Tests

Test Structure:

Example Integration Test:

🎭 END-TO-END TESTING

4.1 E2E Test Scenarios

Test Structure:

Example E2E Test:

⚡ PERFORMANCE TESTING

5.1 Load Testing

Load Test Scenarios:

Load Test Implementation:

5.2 Performance Monitoring

Flutter Performance Tests:

🔒 SECURITY TESTING

6.1 Security Test Scenarios

Test Categories:

Security Test Implementation:

📊 TEST AUTOMATION

7.1 CI/CD Pipeline Tests

GitHub Actions Configuration:

7.2 Test Reporting

Test Coverage Reporting:

📋 TESTING CHECKLIST

Pre-Release Testing:

Continuous Testing:

Production Monitoring:

🛠️ TESTING TOOLS & FRAMEWORKS

Flutter Testing:

Backend Testing:

Performance Testing:

Security Testing:

📚 TESTING BEST PRACTICES

General Principles:

Test Organization:

Test Maintenance:

47 KiB

Raw Permalink Blame History