Commit Graph
100 Commits
Author SHA1 Message Date
syeopite ab4c0a1d3c Fetch channel info from home tab instead of about 2023-11-07 23:53:14 -08:00
syeopite c31908a011 Update params protobuf for channel about tab 2023-11-07 22:19:56 -08:00
syeopite be2feba17c Lint 2023-09-23 09:57:26 -04:00
syeopite a999438ae4 Consistency: rename #add_timestamp_component
Removes the add_ prefix for consistency with the other methods in
WebVTT::Builder
2023-09-23 09:41:43 -04:00
syeopite 4e97d8ad09 Update documentation for WebVTT.build 2023-09-23 09:41:43 -04:00
syeopite d371eb50f2 WebVTT::Builder: rename #line to #cue 2023-09-23 09:41:41 -04:00
syeopite 0cb7d0b441 Refactor Invidious's VTT logic to use WebVtt.build 2023-09-23 09:40:04 -04:00
syeopite 54fa59cbb0 Add method to construct WebVTT files
Similar to JSON.Build
2023-09-23 09:20:20 -04:00
syeopiteandSamantaz Fox 3615bb0e62 Update src/invidious/videos/caption.cr
Co-authored-by: Samantaz Fox <coding@samantaz.fr>
2023-08-24 16:21:05 -07:00
syeopite 1f7592e599 Refactor structure of caption.cr
Rename CaptionsMetadata to Metadata
Nest Metadata under Captions
Unnest LANGUAGES constant from Metadata to main Captions module
2023-08-24 16:00:02 -07:00
syeopite 6b17bb5250 Regression from #4037 | Fix storyboards
PR #4037 introduced a workaround around YouTube's new integrity checks
on streaming URLs. However, the usage of this workaround prevents
storyboard data from being returned by InnerTube.

This commit fixes that by only using the workaround when calling try_fetch_streaming_data
2023-08-08 15:28:01 -07:00
syeopite a81c0f329c Add workaround for storyboards on priv. instances
An upstream problem with videojs-vtt-thumbnails means that URLs gets
joined incorrectly on any instance where `domain`, `external_port` and
`https_only` aren't set.

This commit adds some logic with the 404 handler to mitigate this
problem. This is however only a workaround.

See:
https://github.com/iv-org/invidious/issues/3117
https://github.com/chrisboustead/videojs-vtt-thumbnails/issues/31
2023-08-08 15:13:23 -07:00
syeopite 2f6b2688bb Use workaround for fetching streaming URLs
YouTube appears to be A/B testing some new integrity checks. Adding the
parameter "CgIQBg" to InnerTube player requests appears to workaround
the problem

See https://github.com/TeamNewPipe/NewPipeExtractor/pull/1084
2023-08-06 12:22:22 -07:00
syeopite a8ba02051b Remove(?) lsquic from make and docker files 2023-07-26 07:25:19 -07:00
syeopite c5fe96e936 Remove lsquic from codebase 2023-07-26 07:19:12 -07:00
syeopite 3509752b79 Rename transcript() to get_transcript() in YT API 2023-07-23 16:52:47 -07:00
syeopite e4942b188f Integrate transcript captions into captions API 2023-07-23 14:40:09 -07:00
syeopite caac7e2166 Add method to convert transcripts response to vtt 2023-07-23 14:40:08 -07:00
syeopite 4b3ac1a757 Add method to parse transcript JSON into structs 2023-07-23 14:40:08 -07:00
syeopite 8e18d445a7 Add method to generate params for transcripts api 2023-07-23 14:40:08 -07:00
syeopite 7e5935a9da Rename Caption struct to CaptionMetadata
The Caption object does not actually store any text lines for the
subtitles. Instead it stores the metadata needed to display and fetch
the actual captions from the YT timedtext API.

Therefore it may be wiser to rename the struct to be more reflective of
its current usage as well as the future usage once the current caption
retrival system is replaced via InnerTube's transcript API
2023-07-23 14:40:08 -07:00
syeopite 2e67b90540 Add method to query /youtubei/v1/get_transcript 2023-07-23 14:40:02 -07:00
syeopite a120f143d7 Disable quic by default
See #2577
2021-11-12 04:03:23 -08:00
syeopite 65fbdbff6a Remove of gzip header w/ use_quic config
Continuation of b0f127d4d8
2021-11-12 03:52:50 -08:00
syeopite 6ec4dcfafd Fix handling for maxres thumbnail 2021-11-12 03:47:58 -08:00
syeopite 48191aca6e Fix copy-paste error 2021-11-12 03:47:57 -08:00
syeopite 83556bace2 Allow thumbnail queries with QUIC disabled 2021-11-12 03:47:57 -08:00
syeopite 814c9e6c3a Use https for storyboard image requests 2021-11-12 03:47:57 -08:00
syeopite 547abe17d9 Use https for ggpht requests 2021-11-12 03:47:57 -08:00
syeopite 6b8450558d Allow storyboard queries with QUIC disabled 2021-11-12 03:47:57 -08:00
syeopite c3747c2d49 Allow ggpht queries with QUIC disabled 2021-11-12 03:47:57 -08:00
syeopite 245122104a Respect use_quic param and fix typos 2021-11-12 03:47:57 -08:00
syeopite b0f127d4d8 Fix gzip decompression with HTTP::Client 2021-11-12 03:47:57 -08:00
syeopite d379a36c0e Add compile-time flag to remove code for QUIC 2021-11-12 03:47:50 -08:00
syeopite 437bdedb07 Use env.request.resource for instance switch link 2021-10-26 16:12:25 -07:00
syeopite a531f4f057 Rebase error 2021-10-26 16:12:25 -07:00
syeopite bb396a310e Remove unneeded redirect.invidious.io code in /redirect 2021-10-26 16:12:25 -07:00
syeopite 66e7285108 Only use /redirect when automatically redirecting 2021-10-26 16:12:16 -07:00
syeopite e2bbc9a6fa Switch to #sort_by! in pull_popular_videos job 2021-10-25 01:14:14 -07:00
syeopite 17e481c107 Reduce block verbosity further 2021-10-25 01:14:14 -07:00
syeopite b8f27a42a7 Optimize assigning of on_response_received_endpoints 2021-10-25 01:14:14 -07:00
syeopite 575c66efd3 Remove extra dot between [] from ambea copy-pasting. 2021-10-25 01:14:14 -07:00
syeopite dd8c412abc Fix Style/IsAFilter issues 2021-10-25 01:14:08 -07:00
syeopite e969c1490a Fix Performance/ChainedCallWithNoBang issues 2021-10-25 01:12:26 -07:00
syeopite e91421253e Fix Style/VerboseBlock issues 2021-10-25 01:12:26 -07:00
syeopite 35d15c7c2b Fix Style/VariableNames issues 2021-10-25 01:12:26 -07:00
syeopite 20cb751ff6 Fix Lint/UnusedArgument issues 2021-10-25 01:12:26 -07:00
syeopite 0614b52f03 Fix Lint/RedundantStringCoercion issues 2021-10-25 01:12:19 -07:00
syeopite 585e4617e8 Lazy load (some) images 2021-10-14 08:18:44 -07:00
syeopite 4d44b2c3a4 Handle YT tabs without any content 2021-10-13 11:38:20 -07:00
syeopite b49b5fbda9 Support empty categories 2021-10-13 11:35:31 -07:00
syeopite 4246c7a523 Extract image routes 2021-10-11 13:22:11 -07:00
syeopite 73eabb6ca2 Actually fix lint 2021-10-02 08:53:58 -07:00
syeopite 0c7726d4e1 Update/create versions.yml after dep installation 2021-10-02 06:17:03 -07:00
syeopite 1f1e14fba5 Propagate exceptions from fiber 2021-10-02 06:07:48 -07:00
syeopite 9be8263f26 Use command literal for extraction 2021-10-02 05:52:42 -07:00
syeopite 26b28cea49 Use break instead of short-circuit return 2021-10-01 05:39:23 -07:00
syeopite 23049e026f Improve readabltiy of SearchChannel auto-gen detect 2021-09-28 08:55:02 -07:00
syeopite 9ab242ca2e Optimize routing logic of extract_item(s) funcs 2021-09-28 08:50:23 -07:00
syeopite aa59925374 Rename get_browse_endpoint to get_browse_id 2021-09-28 08:39:00 -07:00
syeopite 43ea8fa706 Convert nil for AuthorFallback to empty strings 2021-09-28 08:25:03 -07:00
syeopite 092b8a4e52 Add documentation to extractors.cr 2021-09-24 21:07:07 -07:00
syeopite e5f07dedbf Typos and tiny styling changes 2021-09-24 21:07:07 -07:00
syeopite ca9eb0d539 Bountiful extractor changes
- Add extract_text to simplify extraction of InnerTube texts
- Add helper extractor methods to reduce repetition in parsing InnerTube
- Change [] more than 2 blocks long to use #dig or #dig?
- Remove useless ?.try blocks for items that always exists
- Add (some) documentation to VideoRendererParser
2021-09-24 21:07:07 -07:00
syeopite 142317c2be Overhaul extractors.cr to use modules 2021-09-24 21:07:07 -07:00
syeopite 3dea670091 Switch to structs in extractors.cr for performance 2021-09-24 21:07:07 -07:00
syeopite 8435e79913 Improve documentation for extract_item(s) funcs 2021-09-24 21:07:07 -07:00
syeopite 30e85b40f9 Fix extract_videos 2021-09-24 21:07:07 -07:00
syeopite be1a43a337 Manually extract category refactor from 1b569bbc99207cae7c20aa285f42477ae361dd30
Also fixes some errors caused by cherry-picking
2021-09-24 21:07:07 -07:00
syeopite abca8f7a7c Rename invidiousitems.cr 2021-09-24 21:07:07 -07:00
syeopite 7b60dac526 Add description_html field to Category
(cherry picked from commit aa8f15f795787113e56473f8e8fd606749a14bdd)
2021-09-24 21:07:07 -07:00
syeopite ea6434662d Change typing of Category contents to only Array
(cherry picked from commit d3384e17f10d0baca70db7993df14100485be9da)
2021-09-24 21:07:06 -07:00
syeopite 0b7a108a59 Move continuation_token out of Category struct
(cherry picked from commit 0e96eda28f25171a0344b972af1852a4d6fc3007)
2021-09-24 21:07:06 -07:00
syeopite 57c63f3598 Rename "items_without_cate_items" to reflect usage 2021-09-24 21:07:06 -07:00
syeopite ae30f32c36 Unpack search items that are embedded in categories
This is a squash of a bunch of commits
cherry-picked commits

Fix category parse error on search

(cherry picked from commit cc02fed4e69f0eb5f19e017173632b3a3f20519f)

Fix category items not being extracted in search

(cherry picked from commit 2605b9c609ff217b5a6ae09d22450596dcad90fc)

Make search not include category items for now

(cherry picked from commit ca4afd59f46b595e3c339f31432cad98a5771ee1)

Change behavior of categories in search results

(cherry picked from commit cc1067561051b1c113b490e79c4a71cd346f7b3f)

Fix missing search results in extraction

(cherry picked from commit abda6840d5bfe58f845128bdd1a3f4916dd3bb84)

Fix miscount of search results

(cherry picked from commit 491e33450eb1300d0234bb33df0d0e78a027114f)
2021-09-24 21:07:06 -07:00
syeopite a50f64f6e9 Add parser for categories (shelfRenderer)
This commit adds a new parser for YT's shelfRenderers which are
typically used to denote different categories.The code for featured
channels parsing has also been moved to use the new parser but some
additional refactoring are needed there.

The ContinuationExtractor has also been improved and is now capable of
extraction continuation data that is packaged under
"appendContinuationItemsAction"

In additional this commit adds some useful helper functions to extract
the current selected tab the continuation token. This is to mainly
reduce code size and repetition.
--
This cherry-picked commit also removes the code for parsing featured
channels present on the original.

(cherry picked from commit 8000d538dbbf1eb9c78e000b1449926ba3b24da9)
2021-09-24 21:07:06 -07:00
syeopite 1323b94b7a Rewrite extract_item and extract_items functions
This commit completely rewrites the extract_item and extract_items
function. Before this commit these two function were an unreadable
mess. The extract_item function was a lengthy if-elsif chain
while the extract_items function contained an incomprehensible
mess of .try, else and ||.

With this commit both of these functions have been pulled into a
separate file with the internal logic being moved to a few classes.

This significantly reduces the size of these two methods, enhances
readability and makes adding new extraction/parse rules much simpler.

See diff for details.

--
This cherry-picked commit also removes the code for parsing featured
channels present on the original.

(cherry picked from commit a027fbf7af1f96dc26fe5a610525ae52bcc40c28)
2021-09-24 21:06:46 -07:00
syeopite 67b01506c9 Update note on video.js in js dependencies file 2021-09-19 15:06:59 -07:00
syeopite b20b802a8d Bump video.js to 7.11.0 2021-09-19 14:56:05 -07:00
syeopite 45a1407144 Downgrade video.js version
Dash qualties can't be changed on anything higher than 7.10.2. This is
likely something wrong with videojs-http-source-selector.
2021-09-18 19:59:43 -07:00
syeopite 4a0359c04e Pass success msg from dep script during compile-time 2021-09-18 14:48:57 -07:00
syeopite 4e629ca858 Use shell command sha1sum for checksum
Crystal doesn't support OpenSSL3, the version Alpine uses. See
https://github.com/iv-org/invidious/pull/2397#issuecomment-922375908
2021-09-18 14:42:41 -07:00
syeopite c32cae00d5 Fix phrasing 2021-09-18 13:30:17 -07:00
syeopite 02431b3f98 Use correct videojs-vtt-thumbnails location in licences.ecr 2021-09-18 13:30:17 -07:00
syeopite 0323202a03 Revert to iv-org fork of silvermine-videojs-quality-selector
Upstream requires at least two additional sources. Whereas Invidious needs it to be
able to display a single additional source for normal (dashless)
qualites. Aka medium and hd720.
2021-09-18 13:30:17 -07:00
syeopite b6670a7e3d Fix minified silvermine-...quality* fetching 2021-09-18 13:30:17 -07:00
syeopite 62c1991b88 Typo 2021-09-18 13:30:17 -07:00
syeopite 3a6085ad31 Readd player.css 2021-09-18 13:30:17 -07:00
syeopite 6e1b62aedf Fix docker
Clone scripts folder to docker container

Copy videojs-dependencies.yml to docker container

Copy assets folder eariler in build process

Copy assets folder from builder
2021-09-18 13:30:17 -07:00
syeopite 2451497b31 Typo 2021-09-18 13:30:17 -07:00
syeopite 6699c80357 Preserve videojs directory for dependency script 2021-09-18 13:30:17 -07:00
syeopite 1af1474d04 Change "VideoJS" in put reports to "Player" 2021-09-18 13:30:17 -07:00
syeopite 06a1d2ac41 Rename fetch_videojs* to fetch_player* 2021-09-18 13:30:16 -07:00
syeopite f47b588b91 Print message when VideoJS deps are satisfied 2021-09-18 13:30:16 -07:00
syeopite 262131f68e Add script to resolve and fetch VideoJS files 2021-09-18 13:30:15 -07:00
syeopite ceae48088e Stop shipping VideoJS files 2021-09-18 13:28:48 -07:00
syeopite 387bddb51b Improve detection and handling of yt redirect links 2021-09-05 13:13:37 -07:00
syeopite a28945273d Propagate replacing yout.be links to /watch to RSS 2021-09-03 03:33:49 -07:00
syeopite fd6f03655e Fix typo causing links to be youtube.com/redirect 2021-09-03 03:30:36 -07:00
syeopite d984a898d4 Remove usage of haltf in /api/v1/channels/:ucid/comments 2021-08-23 17:07:45 -07:00