Commit Graph

2170 Commits

Author SHA1 Message Date
7b2aa5f98e add icon-buttons to playlist items (#2442) 2021-10-02 11:59:33 +00:00
26b28cea49 Use break instead of short-circuit return 2021-10-01 05:39:23 -07:00
23049e026f Improve readabltiy of SearchChannel auto-gen detect 2021-09-28 08:55:02 -07:00
9ab242ca2e Optimize routing logic of extract_item(s) funcs 2021-09-28 08:50:23 -07:00
aa59925374 Rename get_browse_endpoint to get_browse_id 2021-09-28 08:39:00 -07:00
43ea8fa706 Convert nil for AuthorFallback to empty strings 2021-09-28 08:25:03 -07:00
6df85718e6 Apply suggestions from code review
Co-authored-by: Samantaz Fox <coding@samantaz.fr>
2021-09-28 15:23:36 +00:00
9ba3e1cdb4 Decrease channel refresh frequency (1 min -> 1 h)
This is a temporary fix to reduce load on instances with many channels and avoid IP being flagged by Google.
2021-09-26 23:31:14 +02:00
b20f72b963 Use default timeout (5 seconds) for YT pool (#2430) 2021-09-26 23:03:45 +02:00
092b8a4e52 Add documentation to extractors.cr 2021-09-24 21:07:07 -07:00
e5f07dedbf Typos and tiny styling changes 2021-09-24 21:07:07 -07:00
ca9eb0d539 Bountiful extractor changes
- Add extract_text to simplify extraction of InnerTube texts
- Add helper extractor methods to reduce repetition in parsing InnerTube
- Change [] more than 2 blocks long to use #dig or #dig?
- Remove useless ?.try blocks for items that always exists
- Add (some) documentation to VideoRendererParser
2021-09-24 21:07:07 -07:00
142317c2be Overhaul extractors.cr to use modules 2021-09-24 21:07:07 -07:00
3dea670091 Switch to structs in extractors.cr for performance 2021-09-24 21:07:07 -07:00
8435e79913 Improve documentation for extract_item(s) funcs 2021-09-24 21:07:07 -07:00
30e85b40f9 Fix extract_videos 2021-09-24 21:07:07 -07:00
be1a43a337 Manually extract category refactor from 1b569bbc99207cae7c20aa285f42477ae361dd30
Also fixes some errors caused by cherry-picking
2021-09-24 21:07:07 -07:00
abca8f7a7c Rename invidiousitems.cr 2021-09-24 21:07:07 -07:00
7b60dac526 Add description_html field to Category
(cherry picked from commit aa8f15f795787113e56473f8e8fd606749a14bdd)
2021-09-24 21:07:07 -07:00
ea6434662d Change typing of Category contents to only Array
(cherry picked from commit d3384e17f10d0baca70db7993df14100485be9da)
2021-09-24 21:07:06 -07:00
0b7a108a59 Move continuation_token out of Category struct
(cherry picked from commit 0e96eda28f25171a0344b972af1852a4d6fc3007)
2021-09-24 21:07:06 -07:00
57c63f3598 Rename "items_without_cate_items" to reflect usage 2021-09-24 21:07:06 -07:00
ae30f32c36 Unpack search items that are embedded in categories
This is a squash of a bunch of commits
cherry-picked commits

Fix category parse error on search

(cherry picked from commit cc02fed4e69f0eb5f19e017173632b3a3f20519f)

Fix category items not being extracted in search

(cherry picked from commit 2605b9c609ff217b5a6ae09d22450596dcad90fc)

Make search not include category items for now

(cherry picked from commit ca4afd59f46b595e3c339f31432cad98a5771ee1)

Change behavior of categories in search results

(cherry picked from commit cc1067561051b1c113b490e79c4a71cd346f7b3f)

Fix missing search results in extraction

(cherry picked from commit abda6840d5bfe58f845128bdd1a3f4916dd3bb84)

Fix miscount of search results

(cherry picked from commit 491e33450eb1300d0234bb33df0d0e78a027114f)
2021-09-24 21:07:06 -07:00
a50f64f6e9 Add parser for categories (shelfRenderer)
This commit adds a new parser for YT's shelfRenderers which are
typically used to denote different categories.The code for featured
channels parsing has also been moved to use the new parser but some
additional refactoring are needed there.

The ContinuationExtractor has also been improved and is now capable of
extraction continuation data that is packaged under
"appendContinuationItemsAction"

In additional this commit adds some useful helper functions to extract
the current selected tab the continuation token. This is to mainly
reduce code size and repetition.
--
This cherry-picked commit also removes the code for parsing featured
channels present on the original.

(cherry picked from commit 8000d538dbbf1eb9c78e000b1449926ba3b24da9)
2021-09-24 21:07:06 -07:00
1323b94b7a Rewrite extract_item and extract_items functions
This commit completely rewrites the extract_item and extract_items
function. Before this commit these two function were an unreadable
mess. The extract_item function was a lengthy if-elsif chain
while the extract_items function contained an incomprehensible
mess of .try, else and ||.

With this commit both of these functions have been pulled into a
separate file with the internal logic being moved to a few classes.

This significantly reduces the size of these two methods, enhances
readability and makes adding new extraction/parse rules much simpler.

See diff for details.

--
This cherry-picked commit also removes the code for parsing featured
channels present on the original.

(cherry picked from commit a027fbf7af1f96dc26fe5a610525ae52bcc40c28)
2021-09-24 21:06:46 -07:00
6d68fbc31d Fix livestream regex regression caused by #2271
Closes #2352

Special thanks to @WaywardHeart for finding this issue!
2021-09-25 02:47:52 +00:00
86ca568d6d Remove login type button from frontend (#2423) 2021-09-23 08:44:26 +02:00
2b0bb69a4f Fix mixes route (#2421) 2021-09-20 22:39:32 -07:00
87f46a7532 Unregister captcha job (#2390) 2021-09-19 10:30:19 +00:00
e655af251c Try not to log search queries (#2362) 2021-09-16 09:36:22 +00:00
5054510d15 Prevent VR from being initialized in listen mode (#2396) 2021-09-15 01:37:23 +02:00
947fe4fbb3 HTML escape video mimetype
Video mimetype may contain code information between double quotes.
If not properly escaped, it breaks the browser's parser. E.g:
```
type="video/mp4; codecs=" avc1.64001f,="" mp4a.40.2""=""
```

Thank Robin for catching this!
2021-09-13 18:20:11 +02:00
50c8afb525 Handle equirectangular projections for VR (#2379) 2021-09-10 07:42:39 +00:00
a539de4f97 Bump dependencies (#2378)
* Upgrade to Kemal v1.1.0

* Bump postgres driver
2021-09-10 07:42:15 +00:00
8d5a867271 Merge pull request #2380 from syeopite/fix-invidious-showing-yt-redirect-links
Fix invidious showing yt redirect links
2021-09-06 11:15:08 +02:00
387bddb51b Improve detection and handling of yt redirect links 2021-09-05 13:13:37 -07:00
41ba19b615 fix comment replies 2021-09-04 15:37:16 +02:00
a28945273d Propagate replacing yout.be links to /watch to RSS 2021-09-03 03:33:49 -07:00
fd6f03655e Fix typo causing links to be youtube.com/redirect 2021-09-03 03:30:36 -07:00
8b62c05fe2 remove 3gp only from the player (#2376)
+ video quality precedence on default player when js is not enabled
2021-09-03 09:39:11 +02:00
a1001ada47 Properly transform youtu.be links to be /watch routes in comments and descriptions (#2365) 2021-09-01 12:59:47 +02:00
8e3ff79f22 Remove the mention of 'Omar Roth' from the footer 2021-09-01 12:23:50 +02:00
4fcd0964cd Extract API routes (#2271)
* Extract API routes from invidious.cr
* Remove deprecated APIs
  - insights
  - top feed
2021-08-30 18:27:47 +02:00
5005212bec Extract feed routes (#2269)
* Extract feed routes from invidious.cr
* Removes the deprecated route for /feed/top
* Deprecate /view_all_playlist & use /feed/playlists
* Move feed views into their own directory

* Add haltf method to halt current route context
* Change status_code + return blocks to use haltf

* Set appropriate response headers for RSS routes
2021-08-30 16:58:24 +02:00
a279d6f433 Fix livestream parsing URLs (#2356) 2021-08-26 21:02:26 +00:00
fceb8093f1 Use athena-negotiation to detect language through Accept-Language header (#2324)
Detect language through Accept-Language header
2021-08-24 19:59:27 +00:00
d984a898d4 Remove usage of haltf in /api/v1/channels/:ucid/comments 2021-08-23 17:07:45 -07:00
52688106e4 Fix /api/v1/search/suggestions route link 2021-08-23 16:38:29 -07:00
66b45a8fe2 Bountiful changes
- Use haltf in more locations
 - Fix wrong URL params
 - Rename API modules
 - Remove API routing file and move everything to general iv routing
   file
2021-08-23 16:28:30 -07:00
b5d2eb5c70 fetch with innertube api when video is unavailable (#2329)
+ rename some client type to better names
+ fix thirdParty hack
2021-08-16 10:41:16 -07:00