Summing the sizes of each cached file every time is very inefficient.
Instead we can simply store the cache size in an constant and increase
it everytime a file is added into the cache.
Running `crystal spec` without a file argument essentially produces one
big program that combines every single spec file, their imports, and
the files that those imports themselves depend on. Most of the types
within this combined program will get ignored by the compiler due to a
lack of any calls to them from the spec files.
But for some types, partially the HTTP module ones, using them within
the spec files will suddenly make the compiler enable a bunch of
previously ignored code. And those code will suddenly require the
presence of additional types, constants, etc. This not only make it
annoying for getting the specs working but also makes it difficult to
isolate behaviors for testing.
The `static_assets_handler_spec.cr` causes this issue and so will be
marked as an isolated spec for now. In the future all of the tests
should be organized into independent groupings similar to how the
Crystal compiler splits their tests into std, compiler, primitives and
interpreter.
Overriding `#call` or patching out `serve_file_compressed` provides
only minimal benefits over the ease of maintenance granted by only
overriding what we need to for the caching behavior.
Kemal's subclass of the stdlib `HTTP::StaticFileHandler` is not as
maintained as its parent, and so misses out on many enhancements and bug
fixes from upstream, which unfortunately also includes the patches for
security vulnerabilities...
Though this isn't necessarily Kemal's fault since the bulk of the stdlib
handler's logic was done in a single big method, making any changes hard
to maintain. This was fixed in Crystal 1.17.0 where the handler
was refactored into many private methods, making it easier for an
inheriting type to implement custom behaviors while still leveraging
much of the pre-existing code.
Since we don't actually use any of the Kemal specific features added by
`Kemal::StaticFileHandler`, there really isn't a reason to not just
create a new handler based upon the stdlib implementation instead which
will address the problems mentioned above.
This PR implements a new handler which inherits from the stdlib variant
and overrides the helper methods added in Crystal 1.17.0 to add the
caching behavior with minimal code changes. Since this new handler
depends on the code in Crystal 1.17.0, it will only be applied on
versions greater than or equal to 1.17.0. On older versions we'll
fallback to the current monkey patched `Kemal::StaticFileHandler`
This changes its behavior to align with the stdlib variant in that
compression is now delayed till the moment that the server begins to
send a response.
This allows the handler to avoid compressing empty responses,and
safeguards against any double compression of content that may occur
if another handler decides to compressi ts response.
This does however come at the drawback(?) of it now removing
`content-length` headers on requests if it exists; since compression
makes the value inaccurate anyway.
See: https://github.com/crystal-lang/crystal/pull/9625
The exception handling for database connections results in an
`ex` variable which Ameba sees as overshadowing the `ex` used by the
`ex` block arg used to define the HTTP status code 500 handler below.
Although this is a non-issue since the db connection exception handling
will cause Invidious to exit, Ameba's nature as a static checker means
that it isn't aware of this.
The simplest fix without a dirty ameba ignore comment is to rename `ex`
within the Kemal handler block below, since `ex` within a begin rescue
block is a Crystal convention that will also cause Ameba to raise when
not adhered to.
textcaptcha.com seems to be down since April and it does not appear that
service will be restored.
Text captchas can be easily automated using free LLMs, so keeping the
text captcha is more like a gate to create accounts in mass on public
Invidious instances.
It also gives headaches like bots automating account creation to modify
the videos that appear popular page of each instance (since the popular
page is based on the subscriptions of the registered users).
This commit automates the process of documenting the licenses of
Invidious Javascript files through a compile time macro in the
licenses.ecr template file.
This should hopefully help keep the license documentation up-to-date
and allow extensions like LibreJS to always be able to load the latest
Javascript files of Invidious.
Currently only Invidious's first-party Javascript files are supported.
In the future it should be possible to leverage videojs-dependencies.yml
to automatically document the Javascript licenses for
VideoJS and co. as well.
Remove explicit `self.` from #process of parsers
Remove explicit return tuple in get_issue_template
Fix formatting
Move inline issue template style to stylesheet
Use @id in ProblematicTimelineItem xml repr
Fix naming
Prior to this commit, if even a single item fails to parse Invidious
will throw out an error. This means that even if everything else
on a page can be parsed and rendered without issues, the single
problematic item will cause the entire page to be unusable.
This commit gracefully handles parse errors by catching and then
replacing the problematic item with a new "timeline error" object
that represents the parse error. This will allow the rest of the page
to be rendered and an error card that will replace the location of the
problematic item.
Invidious does not currently support non-livestream hls playback
Originally, the HLS manifest check was essentially a boolean:
if the HLS manifest field was present, it was assumed to be a
livestream. Some videos include the HLS Manifest but aren't
livestreams.
In the case where they are livestreams, the video contains a videoType
field with the value "Livestream". In the case that they're normal
videos, the videoType is "Video". This is exposed via the
`video.live_now` property.
This commit just checks that `video.live_now` is true before treating
it as a livestream
The automatic instance redirection has the potential to pick
the same instance the user is currently on. This is especially
prevalent when the instance list is limited in number like how it is
today.
This PR checks the domain of the instance and ensures that it is not
the same as the current instane before redirecting the user to it.
Otherwise, it just sends the user to rediret.invidious.io
Kilt is unmaintained and the ECR templating logic has been
natively integrated into Kemal with the issues previously seen
having been resolved.
This commit is mostly a precursor to support the next Kemal
release which will add the ability to create error handlers for
raised exceptions.
See https://github.com/kemalcr/kemal/pull/688
The http_proxy section was not commented out in the example config
causing Invidious to error out unless an HTTP proxy was configured.
This problem affects new manual installs in which the example config
is copied to create the actual config Invidious uses
Ameba could be built with an older version of Crystal that follows
a different set of formatting rules than the latest version causing
the Lint/Formatting rule to fail when in actuality the code is actually
compliant with the formatting rules in the latest version of Crystal
syeopite