Answer: this is not possible with merely checking URLs, but it is likely that the multimedia files do not change often, so it is likely that having a "do not update" list for multimedia would be more useful.

Instead for text pages, it would be more useful to first get the page creation date being touched. See here and here for reference. (It could be inaccurate however)
In Python there is a solution with urllib

from urllib.request import urlopen
urlopen("http://example.com").headers['last-modified']

Some other people have recommended the use of checksum instead, but that poses a risk on dynamically generated websites (especially with ads) that have content that constantly mutates (e.g. recommended reading lists).

There is no perfect solution, a person would have to make a sound judgement as to see which one is better.

Overwrite only if file changed mode #41

Description

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions