Commit Graph

59 Commits

Author SHA1 Message Date
rr- c366b608da server/search: fix negating complex searches
Entering:

    miko -miko

is a contradiction that shouldn't have been returning any matches, but
it has nonetheless. This change fixes the construction of negated
expressions that use subqueries.
2016-08-28 18:43:05 +02:00
rr- 3c5878cb16 server/tags: improve tag list performance 2016-08-27 22:19:01 +02:00
rr- 06ab98fa70 server/search: fix sort:random breaking tags
Using sqlalchemy's subqueryload to fetch tags works like this:

1. Get basic info about posts with query X
2. Copy query X
3. SELECT all tags WHERE post_id IN (SELECT post_ids FROM query X)
4. Associate the resulting tags with the posts

When original query contains .order_by(func.random()), it looks like
this:

1. SELECT post.* FROM post ORDER BY random() LIMIT 10
2. Copy "ORDER BY random() LIMIT 10"
3. SELECT tag.* FROM tag WHERE tag.post_id IN (
       SELECT id FROM post ORDER BY random() LIMIT 10)
4. Disaster! Each post now has completely arbitrary tags!

To circumvent this, we replace eager loading with lazy loading. This
generates one extra query for each result row, but it has no chance of
producing such anomalies. This behavior is activated only for
queries containing "sort:random" and derivatives so it shouldn't hit
performance too much.
2016-08-27 01:21:59 +02:00
rr- f8e91a10e8 server/search: refactor query factories 2016-08-27 01:19:29 +02:00
rr- 6d26b5c37a server/search: fix sort:random 2016-08-26 23:27:33 +02:00
rr- fa60b42f65 server/search: improve post list performance 2016-08-26 17:57:20 +02:00
rr- 422b99ac8d server/search: add content-checksum 2016-08-26 16:26:06 +02:00
rr- 61d084cc66 server/search: support 'submit:' for anon uploads 2016-08-22 19:45:25 +02:00
rr- 80af79779d server/snapshots: rewrite 2016-08-16 21:51:25 +02:00
rr- 48af5160df server/search: fix negative offsets causing ISE 2016-08-16 17:22:33 +02:00
rr- 9aea55e3d1 server/general: embrace most of PEP8
Ignored only the rules about continuing / hanging indentation.

Also, added __init__.py to tests so that pylint discovers them. (I don't
buy pytest's BS about installing your package.)
2016-08-14 16:44:03 +02:00
rr- b7f2982c9e server/posts: fix relations bidirectionality 2016-07-17 21:14:03 +02:00
rr- c21494be25 server/posts: make relations bidirectional 2016-07-03 18:30:32 +02:00
rr- d0aaf8fa8f server/posts: optimize default sort order 2016-06-14 07:50:22 +02:00
rr- 36eddad424 server/posts: optimize 'posts around' query 2016-06-14 07:50:22 +02:00
rr- 6b68c77e17 server/posts: add relation-count token 2016-06-13 20:09:15 +02:00
rr- 40565e82ae server/posts: change comment-time behavior
Rather than looking at the edit time, now it looks at the creation time.
2016-06-12 14:24:14 +02:00
rr- dfb2e3d027 server/posts: add getting posts around id
Leave undocumented since it add almost no value for the client
applications.
2016-06-08 22:38:35 +02:00
rr- f3bb6c28a1 server/general: cosmetic fixes 2016-06-05 10:39:56 +02:00
rr- 8a5c6f0b31 server/search: fix caching special tokens
special:liked was being reused between users. Now the cache internally
caches object similar to liked:USER.
2016-06-03 19:47:09 +02:00
rr- f0d3589344 server/search: cache by query rather than its text 2016-06-03 19:47:09 +02:00
rr- 59ad5fe402 server/search: refactor, extract parsing 2016-06-03 19:47:09 +02:00
rr- 28009bf46d server/general: fix pylint warnings 2016-06-02 23:39:38 +02:00
rr- 7c7adafd82 server/search: improve error messages 2016-06-02 21:15:29 +02:00
rr- a72f6aa585 server/search: fix searching for invalid numbers 2016-06-02 20:56:28 +02:00
rr- 1034362b84 server/search: fix searching by partial range 2016-06-02 20:56:28 +02:00
rr- 802c7feca0 server/search: add safety search for posts 2016-06-02 13:58:08 +02:00
rr- 446f4d6611 server/search: cache results till non-GET request
250 ms per page --> 30 ms per page (save for the first render). I'd say
it's pretty good
2016-05-31 14:18:50 +02:00
rr- 349f32ccf2 server/search: defer most posts columns 2016-05-30 23:36:58 +02:00
rr- 48bcbbff83 server/search: improve performance 2016-05-30 20:51:52 +02:00
rr- 731c0442e9 server/search: change named token detection
In particular, treat tokens starting with : as anonymous tokens.
2016-05-24 10:29:14 +02:00
rr- 172b956e48 server/search: permit search for tags by alias 2016-05-11 23:47:50 +02:00
rr- 6179f016ae server/search: make searches case insensitive 2016-05-11 23:47:50 +02:00
rr- fcbfa90879 serevr/tags: fix search by post count and category 2016-05-09 22:36:29 +02:00
rr- 339c9a3333 server/tags: reduce number of queries
On a test page with 50 tags, 158 queries were reduced to 3:

1. Get the authenticated user
2. Get tags for given page
3. Count all tags

Looks just about right.
2016-05-08 20:33:16 +02:00
rr- 58964bcdc9 server/posts: add post listing 2016-05-07 21:50:37 +02:00
rr- da5b32feeb server/users: fix hiding email from admins etc 2016-04-29 13:16:16 +02:00
rr- 0b20132a2f server/api + docs/api: organize responses 2016-04-28 18:20:50 +02:00
rr- 0b47957bb9 server/comments: add comment search 2016-04-24 11:56:14 +02:00
rr- a30886cc70 server/search: rename order: to sort: 2016-04-22 19:37:58 +02:00
rr- 46ee9faf72 server/snapshots: add snapshot lists 2016-04-21 19:27:30 +02:00
rr- a926838b90 server/general: rename 'misc' to 'util' 2016-04-20 11:59:46 +02:00
rr- 57b18c6461 server/general: rename 'util' to 'func' 2016-04-20 11:58:17 +02:00
rr- 2e57a0746f server/general: consistently use db.session 2016-04-19 18:10:22 +02:00
rr- fe56e376f6 server/tags: export also tag categories 2016-04-19 15:51:26 +02:00
rr- 1597ae7c5c server/db: refactor session usage 2016-04-18 23:06:51 +02:00
rr- 08271caf32 server/search: rename search criterion 2016-04-17 18:17:18 +02:00
rr- c957befbd0 server/search: simplify virtual token mappings 2016-04-17 18:17:18 +02:00
rr- bc15fb6675 server/posts: add sketch of post table 2016-04-17 16:35:31 +02:00
rr- 0bc45e9c45 server/search: support -min and -max suffixes 2016-04-17 08:31:46 +02:00