Hands-On Firefox AI: Creating Alt Text for Images in PDFs

Following the ability to switch between different search engines from the Address bar, In Nightly, Firefox has an AI feature that can automatically generate image descriptions in PDF documents when users manually add images. The feature isn’t perfect, but it shows promise.

Every major browser is incorporating AI features and services, including chatbots. Google Chrome, Microsoft Edge, Opera, and Brave lead the charge. Google Chrome has announced and is testing new AI features powered by Gemini, including History search, Tab Organization, Tab compare, and more. Microsoft Edge has integrated the AI assistant, Copilot, into its sidebar. Opera has Aria, and Brave has Leo.

Mozilla, primarily reliant on revenue from the Google search deal, has started allowing users to access different chatbots like Gemini, ChatGPT, HuggingChat, and Le Chat Mistral through its Sidebar.

The company isn’t stopping there, aiming to bring “AI-generated alt-text for images inserted into PDFs” – an accessibility feature addressing the needs of the visually impaired and other users with learning disabilities.

Mozilla said the new AI feature will use a fully private on-device AI model. Firefox will add alt text to images in PDFs locally on-device. This feature will be available within Firefox’s built-in PDF Editor and could be expanded for general browsing with screen readers.

Mozilla aims to use AI based on small open-source models for this. This approach requires minimal computing resources.

WindowsReport has tested Firefox’s local alt text generation feature for images in PDF Editor. Here’s how it works:

1. . Open Firefox and a PDF in the browser.
2. Click the >> button in the toolbar and select “Image Alt Settings.“
3. Download the 180 MB AI model and enable “Create alt text automatically.”
4. Click “Close.”
5. Add an image using the toolbar. The AI generates alt text.
6. Edit the description if needed and click “Save.”
7. Review and save the changes to the PDF.

While the feature is in its initial stages and uses a small model, it can generate descriptions that are sometimes accurate. Check the attached examples for details.

Mozilla aims to improve accessibility with this feature, making inaccessible PDF images usable for those who need them most.

What’s your take on Mozilla integrating AI into Firefox PDF Editor to generate alt text for Images? Let us know in the comments below.