SYSTEM CASE STUDY
GradeScope: SZABIST Academic Dashboard
Parsing portal data into actionable insights
Introduction
GradeScope is a local academic dashboard for SZABIST students that transforms portal data into actionable insights. Built with Python, TypeScript, and React, it automates data collection from ZABDESK, parsing attendance, marks, GPA, and subject risk data. The system uses Playwright for browser automation and Streamlit for visualization. The challenge was to create a reliable data pipeline that handles login flows and parsing without exposing credentials. The solution involved a backend sync flow that detects login, extracts necessary data, and stores it locally for analysis.
Problem
Students at SZABIST had to navigate multiple portal pages to track their academic progress, leading to fragmented data and difficulty in identifying at-risk subjects. The existing system required manual data entry and lacked real-time insights, making it hard to make informed decisions about academic performance.
Solution
GradeScope was built to automate data collection from ZABDESK using Playwright, parsing attendance, marks, GPA, and subject risk data. The system uses a FastAPI backend to handle data processing and a React/Vite frontend for visualization. The sync flow detects login, extracts necessary data, and stores it locally. The frontend provides visualizations using Plotly and Streamlit, allowing students to track their academic progress and identify at-risk subjects. The system avoids exposing credentials by using a headless browser for data collection.
Architecture
FastAPI backend handles data processing and API requests React/Vite frontend provides visualization and user interface Playwright automates browser interactions for data collection Local storage of raw and processed data in specified directories Dashboard API endpoints provide access to parsed data Sync flow detects login and initiates data collection Headless browser ensures data collection without exposing credentials Streamlit integration for visualization and user interaction
Key features
- Automated data collection from ZABDESK using Playwright
- Parsing of attendance, marks, GPA, and subject risk data
- Local storage of raw and processed data
- Visualization of academic progress using Plotly and Streamlit
- Sync flow that detects login and extracts data
- Dashboard for tracking academic performance
- Clear data endpoint to reset data
- Headless browser for data collection without exposing credentials
Engineering challenges
- Handling login flows without exposing credentials
- Parsing HTML content for structured data
- Ensuring data consistency across different portal pages
- Managing browser automation with Playwright
- Maintaining data privacy and security
- Creating a reliable sync flow that detects login
- Ensuring the frontend can handle dynamic data updates
Technical highlights
- Assumptions about data parsing logic based on repository structure
- Uncertainty about exact data formats used in portal responses
- Limited information on security measures for data storage
- Assumptions about user interaction flow based on frontend structure
- Uncertainty about exact data parsing logic
Impact summary
GradeScope provides a centralized dashboard for SZABIST students to track their academic progress, identify at-risk subjects, and make informed decisions about their studies. The system reduces the need to navigate multiple portal pages and offers real-time insights into academic performance.
SYSTEM COMPONENTS