Commit Graph

183 Commits

Author SHA1 Message Date
Shyam Sunder 99a69333e6 server/posts/upload: Add youtube-dl functionality
allows for video-based posts to be created by using youtube-dl
on the server. Access is controlled with the 'uploads:use_downloader'
permission.
2020-04-03 13:11:54 -04:00
Shyam Sunder 65202189e1 server/posts/upload: edit default flag behavior
The 'loop' flag will be auto-selected by default on video posts if
the flags parameter is undefined when creating a new post.
2020-03-21 18:25:54 -04:00
Shyam Sunder db0c33bb14 server/func/image_hash: added docstrings to functions 2020-03-13 22:45:11 -04:00
Shyam Sunder 6cc2a91632 server/image_search: add migrations for elasticsearch removal 2020-03-13 22:45:11 -04:00
Shyam Sunder 4c78cf8c47 server/image_search: implement reverse search functionality in postgres
This will remove the dependency on the Elasticsearch database.

The search query is passed currently as raw SQL. Proper implementation
using SQLAlchemy will need custom ORM classed to be made.

Additional config parameter "allow_broken_uploads" has been added.
2020-03-13 22:45:11 -04:00
Shyam Sunder 0e6427d8bc server/tests: use postgresql test database 2020-03-06 18:15:25 -05:00
Shyam Sunder f1a09c21d4 server/func/tag_categories: fixed deprecated SA function call 2020-03-06 10:29:03 -05:00
CorePoint 72e104b145
detect ftypiso5 as mp4 mime type 2020-02-07 12:10:38 +01:00
Shyam Sunder 0ff9f9d5a2 server/func/posts: explicity specify MD5 for post security hash 2020-01-12 12:54:28 -05:00
Shyam Sunder 978a384d9e server/tag-categories: order tag categories alphabetically when requested 2020-01-12 12:18:53 -05:00
neobooru 73c53fa4e2 all: add support for webp images
Includes webp test image
Merges #283
2019-10-08 18:22:47 -04:00
Shyam Sunder 54eab0aa35 server/image-hash: optionally allow for elasticsearch authentication 2019-09-15 16:50:47 -04:00
Shyam Sunder 7a42c7a69b server/tools: add script to check audio flags for posts 2019-07-27 16:32:39 -04:00
Shyam Sunder 4117f63375 server/model/posts: Make post flags a hybrid attribute in model
This should (hopefully) fix #250 and #252
2019-04-22 20:20:19 -04:00
Shyam Sunder b18acf3982 server/func/images: attempt to fix #225 2019-02-11 21:28:02 +01:00
Shyam Sunder 065a466af8 server/func/posts: fix #221 2019-02-11 21:28:02 +01:00
rr- 5ed70b2ec4 server/func/images: work around ffmpeg bug 6375 2019-01-09 21:15:58 +01:00
Shyam Sunder 14377933a7 server/func/posts: transfer flags on merge 2018-12-22 12:31:25 +01:00
Shyam Sunder e80c482891 server/func/images: Fix Unicode Error 2018-12-22 12:31:25 +01:00
Shyam Sunder 8a10fc8ffd server/posts: automatically detect sound in video post uploads 2018-09-24 11:36:13 +02:00
Shyam Sunder 3879c2ec20 server/search: allow searching by post flags 2018-09-24 11:36:13 +02:00
Shyam Sunder 2235a72d2f server+client: added sound flag to video posts 2018-09-24 11:36:13 +02:00
Shyam Sunder 9730aa5c05 client: clean up required Python packages
* Packages that are only used in testing or development
have been moved to `dev-requirements.txt`
* Closes #178
* Minor rewrite to drop the `scikit-image` package, which
saves around 200MB in install size
2018-07-22 14:02:30 +02:00
rr- 1fe22a4d0a server/tag-categories: disallow uppercase colors 2018-07-08 10:10:06 +02:00
ReAnzu 2a69f0193f server/auth: add token authentication
* Users are only authenticated against their password on login,
  and to retrieve a token
* Passwords are wiped from the GUI frontend and cookies
  after login and token retrieval
* Tokens are revoked at the end of the session/logout
* If the user chooses the "remember me" option,
  the token is stored in the cookie
* Tokens correctly delete themselves on logout
* Tokens can expire at user-specified date
* Tokens have their last usage time
* Tokens can have user defined descriptions
* Users can manage login tokens in their account settings
2018-03-25 22:23:29 +02:00
ReAnzu 3f52aceca4 server/users: harden password hashes
- Changed password setup to use libsodium and argon2id (regular SHA256
  hashing for passwords is inadequate as modern GPU's can hash generate
  billions of hashes per second).
- Added code to auto migrate old passwords to the new password_hash if
  the existing password_hash matches either of the legacy password
  generation schemes (SHA1 or SHA256).
- Added migration to support new password_hash format length
- Added column password_revision. This field will default to 0, which
  all passwords will have till they're updated. After that each password
  hash method has a revision.
2018-03-08 23:40:47 +01:00
ReAnzu 7519e071e7 server/posts: deleting a post purges its artifacts
Specifically, its thumbnail and post source.
2018-03-08 23:37:37 +01:00
ReAnzu 12ec43f098 server/posts: auto convert GIFs to WEBMs/MP4s
- Default setting is false for both conversions, as this will require
  additional resources of the server, but is bandwidth friendly for
  viewers
- WEBM conversion is slow, but better quality than MP4 conversion with
  a typically smaller file size
- Tags are copied over from the original upload
- Snapshots are generated for the new auto posts
2018-03-08 07:48:45 +01:00
ReAnzu 4ff8be6a2f server/posts: ignore ffmpeg warnings
Poorly formatted MP4 and WEBM sources can cause ffmpeg to throw a lot
of warnings. However when there is byte ouptut, the generated thumbnail
is valid. Add a bypass for the resize_fill function to allow ffmpeg to
error.
2018-03-08 07:48:44 +01:00
rr- f8c7375b01 server/tags: allow uppercase tag category colors
i.e. colors such as "#FF0000"
2017-10-08 21:38:38 +02:00
rr- 1c4c5c5f91 remove tags.json 2017-10-01 21:48:00 +02:00
rr- 4afece8d50 server/posts: add non-guessable IDs to post URLs 2017-08-24 17:17:09 +02:00
rr- 3c138685ea server/images: handle resizing errors 2017-05-03 12:10:04 +02:00
rr- 4bc58a3c95 server: lint 2017-04-24 23:30:53 +02:00
rr- 467b4a7630 server/tags: fix nondeterministic siblings order 2017-04-24 22:48:11 +02:00
rr- 0014721053 server/tags: fix retrieving many tags 2017-04-19 14:44:54 +02:00
rr- 7044d2aaee server/posts: ignore old elasticsearch results 2017-03-12 18:30:42 +01:00
rr- 5681fd11ef server/net: make the user-agent configurable
Fixes #127
2017-03-03 17:27:23 +01:00
rr- f40a8875c4 server/files: fix import for Py3.5
os.DirEntry is available only from Python3.6+.
2017-02-05 22:38:55 +01:00
rr- ad842ee8a5 server: refactor + add type hinting
- Added type hinting (for now, 3.5-compatible)
- Split `db` namespace into `db` module and `model` namespace
- Changed elastic search to be created lazily for each operation
- Changed to class based approach in entity serialization to allow
  stronger typing
- Removed `required` argument from `context.get_*` family of functions;
  now it's implied if `default` argument is omitted
- Changed `unalias_dict` implementation to use less magic inputs
2017-02-05 16:34:45 +01:00
rr- abf1fc2b2d server: make linters happier 2017-02-03 22:42:14 +01:00
rr- fd30675124 server/image-hash: do not depend on image-match
While I hold this library in great esteem for its excellent work on
implementing the original paper, I have several problems with it:

- as of this commit, it (again) has bug fixes unreleased on pip
- its code is badly structured
    - forces OOP and then proceeds @staticmethod everything
    - bad class design, parameters are repeated in several places
    - terrible contract of make_record() and generate_signature()
    - ambiguous parameters: path vs. image path vs. image content
    - doesn't adhere to PEP-8
- depends on cairo just to render svg images almost no one uses this
  library with
2017-02-03 21:20:52 +01:00
rr- 894cd29511 server/tests: test image hash 2017-02-03 19:53:10 +01:00
rr- f828c375e6 server/posts: fix reverse search late evaluation
Uploading webms caused 'Not an image.' error to be shown, cause
generators are evaluated lazily, so the `catch` never worked.
2017-02-02 21:52:52 +01:00
rr- e92bd2fd80 server/tags: fix getting default category name
No categories? Should have thrown an error rather than returning None.
2017-02-02 20:04:09 +01:00
rr- 07d0b43d4c server/posts: reduce warnings from sqlalchemy
...regarding empty IN() statements
2017-02-02 19:46:03 +01:00
rr- aa1faa3ccb server/image-hash: improve exception handling 2017-02-02 19:46:03 +01:00
rr- f42fbbdc56 server/images: support webm with multiple streams 2017-01-25 17:13:39 +01:00
rr- 0cfc9bcafd server/posts: fix handling corrupt files
In case of a ProcessingError, the image dimensions are set to None. But
after that, they are compared with 0, which resulted in a TypeError.
2017-01-25 17:11:05 +01:00
rr- 8d8165a0d7 server/tags: fix order of aliases in export
fixes #112
2017-01-08 22:29:05 +01:00