Senior Software Engineer (LMS / Python)

Responsibilities:

• LMS Integration Development: Develop and implement secure and efficient methods to access and extract course content from target LMS platforms: Canvas, D2L Brightspace, Blackboard, and Moodle.
• Data Extraction & Preprocessing: Design and build mechanisms to extract various content types (e.g., text, documents, multimedia links, interactive components, videos, assignments, discussions) from LMS courses. This includes handling different file formats and structures. API Utilization: Leverage official LMS APIs where available and appropriate for content access.
• Web Scraping Techniques: Employ advanced web scraping techniques for platforms or content types where direct API access is limited, ensuring resilience to UI changes. Authentication & Authorization: Implement secure authentication flows to access user-authorized course content within the LMS. Assume provision of a service account that the component can use for LMS access.
• Error Handling & Robustness: Develop comprehensive error handling, retry mechanisms, and logging to ensure the scraper is resilient to network issues, LMS changes, and data inconsistencies.
• Scalability & Performance: Design the scraping component to be scalable and performant, capable of handling numerous course extractions efficiently.
• Documentation: Create clear and thorough documentation for the scraping component, including technical specifications, API usage, and troubleshooting guides.
• Collaboration: Work closely with the core development team, particularly with the "Course Content Ingestion Layer" and "LLM Processing Engine" components, to ensure seamless data flow.

Qualifications:

• LMS Expertise: Proven experience working with and understanding the architecture and data structures of major LMS platforms (Canvas, D2L Brightspace, Blackboard, Moodle).
• Web Scraping Proficiency: Strong expertise in web scraping technologies and libraries (e.g., Python with BeautifulSoup, Scrapy, Playwright, Selenium) and an understanding of best practices for ethical and robust scraping.
• API Integration: Solid experience integrating with RESTful APIs, handling authentication (OAuth, API keys), and managing rate limits.
• Programming Languages: Strong proficiency in Python is highly preferred, given its extensive libraries for web scraping and data processing.
• Data Handling: Experience with data parsing, cleaning, and transformation to normalize extracted content into a consistent format.
• Security Best Practices: Understanding of security considerations related to accessing sensitive data and implementing secure coding practices.
• Problem-Solving: Excellent analytical and problem-solving skills to navigate complex LMS structures and handle varying content representations.
• Version Control: Familiarity with Git and collaborative development workflows.