From d12d370e51c0bffa25d96d4b18f74526bde0d98c Mon Sep 17 00:00:00 2001 From: Aria Moradi Date: Wed, 10 Jun 2020 02:08:52 +0430 Subject: [PATCH] [CI SKIP] improve CONTRIBUTING.md (#3443) * update Useful knowledge * update extension workflow * move disclaimer * capitalize sentence beginnings * Update CONTRIBUTING.md * improve debugging info * Update CONTRIBUTING.md * remove info about CatalogSource * Update CONTRIBUTING.md * fix typo * restructure a sentence * Update CONTRIBUTING.md * new unearthed info! * explain SourceFactory, other minor corrections * restructure the text in a more meaningful way * remove redundancy * correct grammar error * Update CONTRIBUTING.md * Update CONTRIBUTING.md * update document header * Update CONTRIBUTING.md * Update CONTRIBUTING.md * Update CONTRIBUTING.md * Update CONTRIBUTING.md * remove disclaimer section * Update CONTRIBUTING.md * Update CONTRIBUTING.md --- CONTRIBUTING.md | 104 +++++++++++++++++++++++++++--------------------- 1 file changed, 59 insertions(+), 45 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 9b22ef2dd..39eb8823e 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -2,10 +2,12 @@ Before you start, please note that the ability to use following technologies is **required** and it's not possible for us to teach you any of them. -* [Kotlin](https://kotlinlang.org/) -* [JSoup](https://jsoup.org/) -* [HTML](https://developer.mozilla.org/en-US/docs/Web/HTML) -* [CSS selectors](https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Selectors) +- basic [Android software development](https://en.wikipedia.org/wiki/Android_software_development) using [Kotlin](https://kotlinlang.org/) +- web scraping + - [OkHttp](https://square.github.io/okhttp/) + - [JSoup](https://jsoup.org/) + - [HTML](https://developer.mozilla.org/en-US/docs/Web/HTML) + - [CSS selectors](https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Selectors) ## Writing an extension @@ -21,9 +23,9 @@ apply plugin: 'com.android.application' apply plugin: 'kotlin-android' ext { - appName = 'Tachiyomi: My catalogue' - pkgNameSuffix = 'lang.mycatalogue' - extClass = '.MyCatalogue' + appName = 'Tachiyomi: My source name' + pkgNameSuffix = 'lang.mysourcename' + extClass = '.MySourceName' extVersionCode = 1 libVersion = '1.2' } @@ -35,13 +37,56 @@ apply from: "$rootDir/common.gradle" | ----- | ----------- | | `appName` | The name of the Android application. By prefixing it with `Tachiyomi: `, it will be easier to locate with an Android package manager. | | `pkgNameSuffix` | A unique suffix added to `eu.kanade.tachiyomi.extension`. The language and the site name should be enough. Remember your catalogue code implementation must be placed in this package. | -| `extClass` | Points to the catalogue class. You can use a relative path starting with a dot (the package name is the base path). This is required for Tachiyomi to instantiate the catalogue. | -| `extVersionCode` | The version code of the catalogue. This must be increased with any change to the implementation and cannot be `0`. | -| `libVersion` | The version of the [extensions library](https://github.com/inorichi/tachiyomi-extensions-lib)* used. | +| `extClass` | Points to the catalogue class. You can use a relative path starting with a dot (the package name is the base path). This is required for Tachiyomi to find and instantiate the catalogue. | +| `extVersionCode` | The version code of the catalogue. This must be a positive integer and increased with any change to the implementation. | +| `libVersion` | The version of the [extensions library](https://github.com/inorichi/tachiyomi-extensions-lib) used. | The catalogue's version name is based off of `libVersion` and `extVersionCode`. With the example used above, the version of the catalogue would be `1.2.1`. -\* Note: this library only contains the method definitions so that the compiler can resolve them. The actual implementation is written in Tachiyomi. +#### Extensions library + +Extensions rely on [extensions-lib](https://github.com/tachiyomiorg/extensions-lib), which simply provides some interfaces and stubs to compile extentsions. The actual implementations are in the [app](https://github.com/inorichi/tachiyomi) inside `eu.kanade.tachiyomi.source` package which you can find it [here](https://github.com/inorichi/tachiyomi/tree/dev/app/src/main/java/eu/kanade/tachiyomi/source), reading the code inside there will help you with writing your extension. + +### general guidelines and extension workflow +- **Extension Main Class** + - The extension Main class which is refrenced and defined by `extClass`(inside `build.gradle`) should be inherited from either `SourceFactory` or one of `Source` children: `HttpSource` or `ParsedHttpSource`. + - `SourceFactory` is used to expose multiple `Source`s, only use it when there's **minor difference** between your target sources or they are essentially mirrors to the same website. + - `HttpSource` as in it's name is for a online http(s) source, but `ParsedHttpSource` has a good model of work which makes writing scrapers for normal aggregator websites much easier and streamlined. (again, you can find the implementation of the stubs in the app as mentioned above) +- The app starts by finding your extension and reads these variables: + +| Field | Description | +| ----- | ----------- | +| `name` | Name to your target source as displayed in the `sources` tab inside the app | +| `id` | identifier of your source, automatically set from `HttpSource`. It should only be manually set if you need to copy an existing autogenerated ID. | +| `supportsLatest` | if `true` the app adds a `latest` button to your extension | +| `baseUrl` | base URL of the target source without any trailing slashes | +| `lang` | as the documentation says "An ISO 639-1 compliant language code (two letters in lower case).", it will be used to catalog your extension | + +- **Popular Manga** + - When user presses on the source name or the `Browse` button on the sources tab, the app calls `fetchPopularManga` with `page=1`, and it returns a `MangasPage` and will continue to call it for next pages, when the user scrolls the manga list and more results must be fetched(until you pass `MangasPage.hasNextPage` as `false` which marks the end of the found manga list) + - While passing magnas here you should at least set `url`, `title` and *`thumbnail_url`; `url` must be unique since it's used to index mangas in the DataBase.(this information will be cached and you will have a chance to update them when `fetchMangaDetails` is called later). + - You may not set `thumbnail_url`, which will make the app call `fetchMangaDetails` over every single manga to show the cover, so it's better to set the thumbnail cover in `fetchPopularManga` if possible. The same is true with latest and search. +- **Latest Manga** + - If `supportsLatest` is set to true the app shows a `Latest` button in front for your extension `name` and when the user taps on it, the app will call `fetchLatestUpdates` and the rest of the flow is similar to what happens with `fetchPopularManga`. + - If `supportsLatest` is set to false no `Latest` button will be shown and `fetchLatestUpdates` and subsequent methods will never be called. +- **Manga Search** + - `getFilterList` will be called to get all filters and filter types. **TODO: explain more about `Filter`** + - when the user searches inside the app, `fetchSearchManga` will be called and the rest of the flow is similar to what happens with `fetchPopularManga`. +- **Manga Details** + - When user taps on a manga and opens it's information Activity `fetchMangaDetails` and `fetchChapterList` will be called the resulting information will be cached. + - `fetchMangaDetails` is called to update a manga's details from when it vas initialized earlier(you may want to parse a manga details page here and fill the rest of the fields) + - Note: During a backup, only `url` and `title` are stored, and to restore the rest of the manga data the app calls `fetchMangaDetails`. so you need to fill the rest of the fields, specially `thumbnail_url`. + - `fetchChapterList` is called to display the chapter list, you want to return a reversed list here(last chapter, first index in the list) +- **Chapter** + - After a chapter list for the manga is fetched, `prepareNewChapter` will be called, after that the chapter will be saved in the app's DataBase and later if the chapter list changes the app will loose any references to the chapter(but chapter files will still be in the device storage) +- **Chapter Pages** + - When user opens a chapter, `fetchPageList` will be called and it will return a list of `Page` + - While a chapter is open the reader will call `fetchImageUrl` to get URLs for each page of the manga +- **Notes** + - Some time while you are writing code, you may find no use for some inherited methods, if so just override them and throw exceptions: `throw Exception("Not used")` + - You probably will find `getUrlWithoutDomain` useful when parsing the target source URLs. + - If possible try to stick to the general workflow from`ParsedHttpSource` and `HttpSource`, breaking them may cause you more headache than necessary. + - When reading the code documentation it helps to follow the subsequent called methods in the the default implementation from the `app`, while trying to grasp the general workflow. ### Additional dependencies @@ -59,11 +104,6 @@ Notice that we're using `compileOnly` instead of `implementation`, since the app ### Core stubs and libraries -#### Extensions library - -Extensions rely on stubs defined in [tachiyomi-extensions-lib](https://github.com/tachiyomiorg/extensions-lib), which simply provides some interfaces for compiling extensions. These interfaces match what's found in the main Tachiyomi app. The exact version used is configured with `libVersion`. The latest version should be preferred. - - #### Duktape stub [`duktape-stub`](https://github.com/inorichi/tachiyomi-extensions/tree/master/lib/duktape-stub) provides stubs for using Duktape functionality without pulling in the full library. Functionality is bundled into the main Tachiyomi app. @@ -84,35 +124,6 @@ dependencies { } ``` -### Useful knowledge - -- An extension should at least extend the [`ParsedHttpSource`](https://github.com/inorichi/tachiyomi-extensions-lib/blob/master/library/src/main/java/eu/kanade/tachiyomi/source/online/ParsedHttpSource.kt) class. -- Do not override id!! It will auto-generate based on the name of the extension and the language. -- Set the thumbnail cover when possible. When parsing the list of manga during latest, search, browse. If not the site will get a new request for every manga that doesn't have a cover shown, even if the user doesnt click into the manga. - - - -#### Flow of the extensions -The structure for an extension is very strict. In the future 1.x release this will be less strict but until then this has caused some issues when some sites don't quite fit the model. There are required overrides but you can override the calling methods if you need more general control. This will go from the highest level method to the lowest level for browse/popular, it is the same but different method names for search and latest. -##### Browse (Aka Popular Manga) -- fetchPopularManga (Optional to override) - - This method takes the results from a manga listing page and parses it. -- popularMangaRequest (Must be overridden) - - The GET/POST for the HTML Page of the manga listings. -- popularMangaParse (Optional to override) - - Parses the manga listing page returns a boolean if has another page, and the manga objects as MangasPage. -- popularMangaSelector (must be overridden) - - jsoup CSS selector to select the list of the manga. -- popularMangaFromElement (must be overriden) - - jsoup selectors to parse the individual manga html on the page (most sites this is just link, title, cover url) -- popularMangaNextPageSelector (must be overridden) - - jsoup CSS selector to see if there is a another page after current one. - - This will provide the initial viewing once a user clicks into a manga you will need to override mangaDetailsParse and this is where you need to parse the actual manga site's manga page and parse the standard info (title, author, description etc etc). - - ###### Note: - Must be overriden even if you override the parent method and it's not being called anymore (for example I override popularMangaParse and don't need the popularMangaNextPage selector. I would just override in the extension and throw a not used exception). - ## Running @@ -131,6 +142,9 @@ And for a release build of Tachiyomi: ``` -W -S -n eu.kanade.tachiyomi/eu.kanade.tachiyomi.ui.main.MainActivity -a eu.kanade.tachiyomi.SHOW_CATALOGUES ``` +## Debugging +- You are encouraged to clone the app itself and make your own debug build, then you can attach Android Studio's debugger to the app, and then you can print logs in the code and debug your extension using `Logcat`([link](https://developer.android.com/studio/debug/am-logcat) to Android Studio documentation for `Logcat`) +- Directly debugging your extension(steeping through your extension code) is not possible, but if you keep both projects(tachiyomi and tachiyomi-extensions) open in Android Studio, you can debug the app itself and hence calls to your code. ## Building