Challenges of Archiving the Web

The web is a technically complex, constantly changing environment.

The Complexity of the Web

The most popular web sites are created by dozens of engineers at large corporations. These sites might look simple but are in fact technically complicated. Additionally, it is not uncommon for such sites to frequently introduce significant technical changes that are not even noticeable to regular users. Smaller teams and single web authors also can have the ability to create technically sophisticated or unusual websites. This complexity makes web archiving difficult.

The Best-Effort Approach

Web archiving with Conifer always works on a best effort basis: we’re doing what we can in a large, diverse web.

Conifer strives to give our users an easy way to create and access quality web collections, where all images load, all videos play, and all social media posts, map widgets, and so forth function as expected. Enabling the creation of captures that faithfully reflect the interactivity of these items require constant maintenance of the web archiving software.

While the majority of our on-going efforts are dedicated to enabling the capture of popular sites, there will always be sites that we have yet to develop a technical capture strategy on. Furthermore, even when a solution to capture a particular site or component has been figured out, the site in question might deploy an update that could bring us back to the drawing board at any time.

How You Can Help

As a user, you can help our efforts by reporting bugs, contributing to the user guide, and sharing your work to help other users.

If you have development skills, consider contributing to the Webrecorder open source toolchain that Conifer is built upon.

Join our mailing list to keep up with the Conifer community.