Project Suki: refactors and fixes (#19513)

* refactor(reportErrorToUser): enhance reportErrorToUser

* refactor(DataExtractor): add unexpectedErrorCatchingLazy

Makes it easier to quickly find and fix unexpected errors

* refactor(BookDetail): attempt at making BookDetail more extensible

Also fixes a rare bug that would throw a NoSuchElementException when the status or origin fields weren't found in the details table.

* refactor(mangaDetailsParse): refactor mangaDetailsParse to follow BookDetail's refactor

* chore(reportErrorToUser): Review reportErrorToUser messages

* refactor(Search): completely separate simple and smart search

create SmartBookSearchHandler as an attempt to speed up search by wasting less resources on unnecessary multiple normalization and reinitialization of resources via ThreadLocal

* chore(build): bumped extVersionCode to 3

* refactor(activities): Add activities to handle /book and /read URLs

Create a MangasPage with only a single Manga present (unfortunately needs to fetch manga details as title can't be inferred just by bookid)
Group activities in "activities" package for clarity

* fix(KDoc): fix Cannot resolve symbol

* chore: Update README and CHANGELOG

* chore: Add a bit of documentation to SmartBookSearchHandler

* feat: Handle /book and /read urls as search query

* chore(CHANGELOG): entries incorrectly listed as PUBLISHING_FINISHED now are correctly listed as COMPLETED

* chore(README): expanded README
This commit is contained in:
Federico d'Alonzo 2023-12-31 22:57:50 +01:00 committed by GitHub
parent 3a213c1ca8
commit 40c354f4d0
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
12 changed files with 788 additions and 272 deletions

View File

@ -2,9 +2,10 @@
<manifest xmlns:android="http://schemas.android.com/apk/res/android"> <manifest xmlns:android="http://schemas.android.com/apk/res/android">
<application> <application>
<!-- The Activity that will handle intents with URLs pertaining to ProjectSuki --> <!-- The Activities that will handle intents with URLs pertaining to ProjectSuki -->
<!-- this one handles projectsuki.com/search URLs -->
<activity <activity
android:name=".all.projectsuki.ProjectSukiSearchUrlActivity" android:name=".all.projectsuki.activities.ProjectSukiSearchUrlActivity"
android:excludeFromRecents="true" android:excludeFromRecents="true"
android:exported="true" android:exported="true"
android:theme="@android:style/Theme.NoDisplay"> android:theme="@android:style/Theme.NoDisplay">
@ -30,5 +31,41 @@
<data android:pathPattern="/search.*" /> <data android:pathPattern="/search.*" />
</intent-filter> </intent-filter>
</activity> </activity>
<activity
android:name=".all.projectsuki.activities.ProjectSukiBookUrlActivity"
android:excludeFromRecents="true"
android:exported="true"
android:theme="@android:style/Theme.NoDisplay">
<intent-filter>
<action android:name="android.intent.action.VIEW" />
<category android:name="android.intent.category.DEFAULT" />
<category android:name="android.intent.category.BROWSABLE" />
<data android:scheme="http" />
<data android:scheme="https" />
<data android:host="projectsuki.com" />
<data android:pathPattern="/book/.*" />
</intent-filter>
</activity>
<activity
android:name=".all.projectsuki.activities.ProjectSukiReadUrlActivity"
android:excludeFromRecents="true"
android:exported="true"
android:theme="@android:style/Theme.NoDisplay">
<intent-filter>
<action android:name="android.intent.action.VIEW" />
<category android:name="android.intent.category.DEFAULT" />
<category android:name="android.intent.category.BROWSABLE" />
<data android:scheme="http" />
<data android:scheme="https" />
<data android:host="projectsuki.com" />
<data android:pathPattern="/read/.*" />
</intent-filter>
</activity>
</application> </application>
</manifest> </manifest>

View File

@ -1,3 +1,12 @@
## Version 1.4.3
- entries incorrectly listed as PUBLISHING_FINISHED now are correctly listed as COMPLETED
- fixes a rare bug that would throw a NoSuchElementException when the status or origin fields
weren't found in the details table.
- separate Smart and Simple search in code to make it easier to debug and extend it in the future
- add activities to handle /book and /read URLs (search/ was already present)
- handle /book and /read urls as search query
## Version 1.4.2 ## Version 1.4.2
- Improved search feature - Improved search feature

View File

@ -1,9 +1,22 @@
# Project Suki # Project Suki
### Issues
Go check out our general FAQs and Guides over at Go check out our general FAQs and Guides over at
[Extension FAQ](https://tachiyomi.org/help/faq/#extensions) or [Extension FAQ](https://tachiyomi.org/help/faq/#extensions) or
[Getting Started](https://tachiyomi.org/help/guides/getting-started/#installation). [Getting Started](https://tachiyomi.org/help/guides/getting-started/#installation).
If you still don't find the answer you're looking for you're welcome to open an If you still don't find the answer you're looking for you're welcome to open an
[issue](https://github.com/tachiyomiorg/tachiyomi-extensions/issues) [issue](https://github.com/tachiyomiorg/tachiyomi-extensions/issues)
and mention [me](https://github.com/npgx/) *in the issue*. **if one isn't open already** and mention [me](https://github.com/npgx/) *in the issue*.
### Usage Tips
- If you know the bookID of a manga you want to search for, you can search for it directly by typing
`book/<bookID>` in the search bar in the Tachiyomi app, this works for `read/<bookID>` too, with
or without `http(s)://projectsuki.com`
(you can copy the link of a book or chapter into the search bar).
- Inside the extension settings (Extensions > ProjectSuki > Gear icon right of "Multi") you can
whitelist and blacklist languages, instructions are provided there.
There you can also provide a custom
[User-Agent](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/User-Agent) if necessary.

View File

@ -6,7 +6,7 @@ ext {
extName = 'Project Suki' extName = 'Project Suki'
pkgNameSuffix = 'all.projectsuki' pkgNameSuffix = 'all.projectsuki'
extClass = '.ProjectSuki' extClass = '.ProjectSuki'
extVersionCode = 2 extVersionCode = 3
} }
dependencies { dependencies {

View File

@ -9,9 +9,9 @@ import org.jsoup.select.Elements
import java.text.SimpleDateFormat import java.text.SimpleDateFormat
import java.util.Calendar import java.util.Calendar
import java.util.Date import java.util.Date
import java.util.EnumMap
import java.util.Locale import java.util.Locale
import java.util.TimeZone import java.util.TimeZone
import kotlin.properties.PropertyDelegateProvider
/** /**
* @see EXTENSION_INFO Found in ProjectSuki.kt * @see EXTENSION_INFO Found in ProjectSuki.kt
@ -23,6 +23,34 @@ internal typealias BookID = String
internal typealias ChapterID = String internal typealias ChapterID = String
internal typealias ScanGroup = String internal typealias ScanGroup = String
/**
* Creates a [delegate provider](https://kotlinlang.org/docs/delegated-properties.html#providing-a-delegate)
* that will return a [Lazy] where the [initializer] is wrapped by a try/catch block that will catch all exceptions
* that aren't a [ProjectSukiException] and constructing a [reportErrorToUser] with a locationHint.
*/
internal fun <R> unexpectedErrorCatchingLazy(mode: LazyThreadSafetyMode = LazyThreadSafetyMode.SYNCHRONIZED, initializer: () -> R): PropertyDelegateProvider<Any?, Lazy<R>> {
return PropertyDelegateProvider { thisRef, property ->
lazy(mode) {
try {
initializer()
} catch (exception: Exception) {
if (exception !is ProjectSukiException) {
val locationHint = buildString {
when (thisRef) {
null -> append("<root>")
else -> append(thisRef::class.simpleName)
}
append('.')
append(property.name)
}
reportErrorToUser(locationHint) { """Unexpected ${exception::class.simpleName}: ${exception.message ?: "<no message>"}""" }
}
throw exception
}
}
}
}
/** /**
* Gets the thumbnail image for a particular [bookID], [extension] if needed and [size]. * Gets the thumbnail image for a particular [bookID], [extension] if needed and [size].
* *
@ -45,14 +73,14 @@ internal fun bookThumbnailUrl(bookID: BookID, extension: String, size: UInt? = n
} }
/** /**
* Finds the closest common parent between 2 or more [elements]. * Finds the nearest common parent between 2 or more [elements] (will return null if [elements].size < 2).
* *
* If all [elements] are the same element, it will return the element itself. * If all [elements] are the same element, it will return the element itself.
* *
* Returns null if the [elements] are not in the same [Document]. * Returns null if the [elements] are not in the same hierarchy (no common parent, e.g. not in the same [Document]).
*/ */
internal fun commonParent(vararg elements: Element): Element? { internal fun nearestCommonParent(elements: Collection<Element>): Element? {
require(elements.size > 1) { "elements must have more than 1 element" } if (elements.size < 2) return null
val parents: List<Iterator<Element>> = elements.map { it.parents().reversed().iterator() } val parents: List<Iterator<Element>> = elements.map { it.parents().reversed().iterator() }
var lastCommon: Element? = null var lastCommon: Element? = null
@ -81,14 +109,10 @@ internal fun commonParent(vararg elements: Element): Element? {
internal data class SwitchingPoint(val left: Int, val right: Int, val leftState: Boolean, val rightState: Boolean) { internal data class SwitchingPoint(val left: Int, val right: Int, val leftState: Boolean, val rightState: Boolean) {
init { init {
if (left + 1 != right) { if (left + 1 != right) {
reportErrorToUser { reportErrorToUser { "invalid SwitchingPoint: ($left, $right)" }
"invalid SwitchingPoint: ($left, $right)"
}
} }
if (leftState == rightState) { if (leftState == rightState) {
reportErrorToUser { reportErrorToUser { "invalid SwitchingPoint: ($leftState, $rightState)" }
"invalid SwitchingPoint: ($leftState, $rightState)"
}
} }
} }
} }
@ -133,7 +157,7 @@ class DataExtractor(val extractionElement: Element) {
private val url: HttpUrl = extractionElement.ownerDocument()?.location()?.toHttpUrlOrNull() ?: reportErrorToUser { private val url: HttpUrl = extractionElement.ownerDocument()?.location()?.toHttpUrlOrNull() ?: reportErrorToUser {
buildString { buildString {
append("DataExtractor class requires a \"from\" element ") append("DataExtractor class requires an \"extractionElement\" element ")
append("that possesses an owner document with a valid absolute location(), but ") append("that possesses an owner document with a valid absolute location(), but ")
append(extractionElement.ownerDocument()?.location()) append(extractionElement.ownerDocument()?.location())
append(" was found!") append(" was found!")
@ -151,7 +175,7 @@ class DataExtractor(val extractionElement: Element) {
* JSoup's [Element.attr] methods supports the special `abs:<attribute>` syntax when working with relative URLs. * JSoup's [Element.attr] methods supports the special `abs:<attribute>` syntax when working with relative URLs.
* It is simply a shortcut to [Element.absUrl], which uses [Document.baseUri]. * It is simply a shortcut to [Element.absUrl], which uses [Document.baseUri].
*/ */
val allHrefAnchors: Map<Element, HttpUrl> by lazy { val allHrefAnchors: Map<Element, HttpUrl> by unexpectedErrorCatchingLazy {
buildMap { buildMap {
extractionElement.select("a[href]").forEach { a -> extractionElement.select("a[href]").forEach { a ->
val href = a.attr("abs:href") val href = a.attr("abs:href")
@ -168,7 +192,7 @@ class DataExtractor(val extractionElement: Element) {
* *
* Meaning this property contains only elements that redirect to a Project Suki URL. * Meaning this property contains only elements that redirect to a Project Suki URL.
*/ */
val psHrefAnchors: Map<Element, HttpUrl> by lazy { val psHrefAnchors: Map<Element, HttpUrl> by unexpectedErrorCatchingLazy {
allHrefAnchors.filterValues { url -> allHrefAnchors.filterValues { url ->
url.host.endsWith(homepageUrl.host) url.host.endsWith(homepageUrl.host)
} }
@ -195,7 +219,7 @@ class DataExtractor(val extractionElement: Element) {
* This has the disadvantage of making distinguishing between the different elements in a single page a nightmare, * This has the disadvantage of making distinguishing between the different elements in a single page a nightmare,
* but luckly we don't need to do that for the purposes of a Tachiyomi extension. * but luckly we don't need to do that for the purposes of a Tachiyomi extension.
*/ */
val books: Set<PSBook> by lazy { val books: Set<PSBook> by unexpectedErrorCatchingLazy {
buildSet { buildSet {
data class BookUrlContainerElement(val container: Element, val href: HttpUrl, val matchResult: PathMatchResult) data class BookUrlContainerElement(val container: Element, val href: HttpUrl, val matchResult: PathMatchResult)
@ -220,7 +244,7 @@ class DataExtractor(val extractionElement: Element) {
.filter { it.parents().none { p -> p.tag().normalName() == "small" } } .filter { it.parents().none { p -> p.tag().normalName() == "small" } }
.map { it.ownText() } .map { it.ownText() }
.filter { !it.equals("show more", ignoreCase = true) } .filter { !it.equals("show more", ignoreCase = true) }
.firstOrNull() ?: reportErrorToUser { "Could not determine title for $bookID" } .firstOrNull() ?: reportErrorToUser("DataExtractor.books") { "Could not determine title for $bookID" }
add( add(
PSBook( PSBook(
@ -237,10 +261,10 @@ class DataExtractor(val extractionElement: Element) {
} }
} }
/** Utility class that extends [PSBook], by providing a [detailsTable], [alertData] and [description]. */ /** Utility class that extends [PSBook], by providing a [details], [alertData] and [description]. */
data class PSBookDetails( data class PSBookDetails(
val book: PSBook, val book: PSBook,
val detailsTable: EnumMap<BookDetail, String>, val details: Map<BookDetail, BookDetail.ProcessedData>,
val alertData: List<String>, val alertData: List<String>,
val description: String, val description: String,
) { ) {
@ -254,37 +278,122 @@ class DataExtractor(val extractionElement: Element) {
* The process for extracting the details is described in the KDoc for [bookDetails]. * The process for extracting the details is described in the KDoc for [bookDetails].
*/ */
@Suppress("RegExpUnnecessaryNonCapturingGroup") @Suppress("RegExpUnnecessaryNonCapturingGroup")
enum class BookDetail(val display: String, val regex: Regex, val elementProcessor: (Element) -> String = { it.text() }) { sealed class BookDetail {
ALT_TITLE("Alt titles:", """(?:alternative|alt\.?) titles?:?""".toRegex(RegexOption.IGNORE_CASE)),
AUTHOR("Authors:", """authors?:?""".toRegex(RegexOption.IGNORE_CASE)), open fun tryFind(extractor: DataExtractor): Collection<Element> = emptyList()
ARTIST("Artists:", """artists?:?""".toRegex(RegexOption.IGNORE_CASE)),
STATUS("Status:", """status:?""".toRegex(RegexOption.IGNORE_CASE)), abstract val regex: Regex
ORIGIN("Origin:", """origin:?""".toRegex(RegexOption.IGNORE_CASE)), fun process(element: Element): ProcessedData = ProcessedData(label(element), detailsData(element))
RELEASE_YEAR("Release year:", """release(?: year):?""".toRegex(RegexOption.IGNORE_CASE)), abstract fun label(element: Element?): String
USER_RATING( abstract fun detailsData(element: Element): String
"User rating:",
"""user ratings?:?""".toRegex(RegexOption.IGNORE_CASE), data class ProcessedData(val label: String, val detailData: String)
elementProcessor = { ratings ->
val rates = when { object AltTitle : BookDetail() {
ratings.id() != "ratings" -> 0 override val regex: Regex = """(?:alternative|alt\.?) titles?:?""".toRegex(RegexOption.IGNORE_CASE)
else -> ratings.children().count { it.hasClass("text-warning") } override fun label(element: Element?) = "Alt titles:"
override fun detailsData(element: Element): String = element.text()
} }
when (rates) { object Author : BookDetail() {
override fun tryFind(extractor: DataExtractor): Collection<Element> = extractor.psHrefAnchors.filter { (_, url) ->
url.queryParameterNames.contains("author")
}.keys
override val regex: Regex = """authors?:?""".toRegex(RegexOption.IGNORE_CASE)
override fun label(element: Element?) = "Authors:"
override fun detailsData(element: Element): String = element.text()
}
object Artist : BookDetail() {
override fun tryFind(extractor: DataExtractor): Collection<Element> = extractor.psHrefAnchors.filter { (_, url) ->
url.queryParameterNames.contains("artist")
}.keys
override val regex: Regex = """artists?:?""".toRegex(RegexOption.IGNORE_CASE)
override fun label(element: Element?) = "Artists:"
override fun detailsData(element: Element): String = element.text()
}
object Status : BookDetail() {
override fun tryFind(extractor: DataExtractor): Collection<Element> = extractor.psHrefAnchors.filter { (_, url) ->
url.queryParameterNames.contains("status")
}.keys
override val regex: Regex = """status:?""".toRegex(RegexOption.IGNORE_CASE)
override fun label(element: Element?) = "Status:"
override fun detailsData(element: Element): String = element.text()
}
object Origin : BookDetail() {
override fun tryFind(extractor: DataExtractor): Collection<Element> = extractor.psHrefAnchors.filter { (_, url) ->
url.queryParameterNames.contains("origin")
}.keys
override val regex: Regex = """origin:?""".toRegex(RegexOption.IGNORE_CASE)
override fun label(element: Element?) = "Origin:"
override fun detailsData(element: Element): String = element.text()
internal val koreaRegex: Regex = """kr|korea\s*(?:\(south\))?""".toRegex(RegexOption.IGNORE_CASE)
internal val chinaRegex: Regex = """kr|korea\s*(?:\(south\))?""".toRegex(RegexOption.IGNORE_CASE)
internal val japanRegex: Regex = """kr|korea\s*(?:\(south\))?""".toRegex(RegexOption.IGNORE_CASE)
}
object ReleaseYear : BookDetail() {
override val regex: Regex = """release(?: year):?""".toRegex(RegexOption.IGNORE_CASE)
override fun label(element: Element?) = "Release year:"
override fun detailsData(element: Element): String = element.text()
}
object UserRating : BookDetail() {
override fun tryFind(extractor: DataExtractor): Collection<Element> = extractor.extractionElement.select("#ratings")
override val regex: Regex = """user ratings?:?""".toRegex(RegexOption.IGNORE_CASE)
override fun label(element: Element?) = "User rating:"
override fun detailsData(element: Element): String {
val rates = when {
element.id() != "ratings" -> 0
else -> element.children().count { it.hasClass("text-warning") }
}
return when (rates) {
in 1..5 -> "$rates/5" in 1..5 -> "$rates/5"
else -> "?/5" else -> "?/5"
} }
}, }
), }
VIEWS("Views:", """views?:?""".toRegex(RegexOption.IGNORE_CASE)),
OFFICIAL("Official:", """official:?""".toRegex(RegexOption.IGNORE_CASE)), object Views : BookDetail() {
PURCHASE("Purchase:", """purchase:?""".toRegex(RegexOption.IGNORE_CASE)), override val regex: Regex = """views?:?""".toRegex(RegexOption.IGNORE_CASE)
GENRE("Genres:", """genre(?:\(s\))?:?""".toRegex(RegexOption.IGNORE_CASE)), override fun label(element: Element?) = "Views:"
; override fun detailsData(element: Element): String = element.text()
}
object Official : BookDetail() {
override val regex: Regex = """official:?""".toRegex(RegexOption.IGNORE_CASE)
override fun label(element: Element?) = "Official:"
override fun detailsData(element: Element): String = element.text()
}
object Purchase : BookDetail() {
override val regex: Regex = """purchase:?""".toRegex(RegexOption.IGNORE_CASE)
override fun label(element: Element?) = "Purchase:"
override fun detailsData(element: Element): String = element.text()
}
object Genre : BookDetail() {
override fun tryFind(extractor: DataExtractor): Collection<Element> = extractor.psHrefAnchors.filter { (_, url) ->
url.matchAgainst(genreSearchUrlPattern).doesMatch
}.keys
override val regex: Regex = """genre(?:\(s\))?:?""".toRegex(RegexOption.IGNORE_CASE)
override fun label(element: Element?) = "Genres:"
override fun detailsData(element: Element): String = element.text()
}
companion object { companion object {
private val values = values().toList() val all: List<BookDetail> = listOf(AltTitle, Author, Artist, Status, Origin, ReleaseYear, UserRating, Views, Official, Purchase, Genre)
fun from(type: String): BookDetail? = values.firstOrNull { it.regex.matches(type) } fun from(type: String): BookDetail? = all.firstOrNull { it.regex.matches(type) }
} }
} }
@ -296,15 +405,15 @@ class DataExtractor(val extractionElement: Element) {
* found in the book main page, using generalized heuristics: * found in the book main page, using generalized heuristics:
* *
* First the algorithm looks for known entries in the "table" by looking for * First the algorithm looks for known entries in the "table" by looking for
* the [Status][BookDetail.STATUS] and [Origin][BookDetail.ORIGIN] fields. * the [Status][BookDetail.Status] and [Origin][BookDetail.Origin] fields.
* This is possible because these elements redirect to the [search](https://projectsuki.com/search) * This is possible because these elements redirect to the [search](https://projectsuki.com/search)
* page with "status" and "origin" queries. * page with "status" and "origin" queries.
* *
* The [commonParent] between the two elements is found and the table is subsequently analyzed. * The [nearestCommonParent] between the two elements is found and the table is subsequently analyzed.
* If this method fails, at least the [Author][BookDetail.AUTHOR], [Artist][BookDetail.ARTIST] and [Genre][BookDetail.GENRE] * If this method fails, at least the [Author][BookDetail.Author], [Artist][BookDetail.Artist] and [Genre][BookDetail.Genre]
* details are found via URLs. * details are found via URLs.
* *
* An extra [Genre][BookDetail.GENRE] is added when possible: * An extra [Genre][BookDetail.Genre] is added when possible:
* - Origin: "kr" -> Genre: "Manhwa" * - Origin: "kr" -> Genre: "Manhwa"
* - Origin: "cn" -> Genre: "Manhua" * - Origin: "cn" -> Genre: "Manhua"
* - Origin: "jp" -> Genre: "Manga" * - Origin: "jp" -> Genre: "Manga"
@ -313,36 +422,24 @@ class DataExtractor(val extractionElement: Element) {
* *
* The description is expanded with all this information too. * The description is expanded with all this information too.
*/ */
val bookDetails: PSBookDetails by lazy { val bookDetails: PSBookDetails by unexpectedErrorCatchingLazy {
val match = url.matchAgainst(bookUrlPattern) val match = url.matchAgainst(bookUrlPattern)
if (!match.doesMatch) reportErrorToUser { "cannot extract book details: $url" } if (!match.doesMatch) reportErrorToUser { "cannot extract book details: $url" }
val bookID = match["bookid"]!!.value val bookID = match["bookid"]!!.value
val authors: Map<Element, HttpUrl> = psHrefAnchors.filter { (_, url) -> fun tryFindDetailsTable(): Element? {
url.queryParameterNames.contains("author") val found: Map<BookDetail, Collection<Element>> = BookDetail.all
.associateWith { it.tryFind(extractor = this) }
.filterValues { it.isNotEmpty() }
return nearestCommonParent(found.values.flatMapTo(LinkedHashSet()) { it })
} }
val artists: Map<Element, HttpUrl> = psHrefAnchors.filter { (_, url) -> val detailsTable: Element? = tryFindDetailsTable()
url.queryParameterNames.contains("artist") val rows: List<Element> = detailsTable?.children()?.toList() ?: emptyList()
} val details: MutableMap<BookDetail, BookDetail.ProcessedData> = LinkedHashMap()
val status: Map.Entry<Element, HttpUrl> = psHrefAnchors.entries.single { (_, url) -> for (row in rows) {
url.queryParameterNames.contains("status")
}
val origin: Map.Entry<Element, HttpUrl> = psHrefAnchors.entries.single { (_, url) ->
url.queryParameterNames.contains("origin")
}
val genres: Map<Element, HttpUrl> = psHrefAnchors.filter { (_, url) ->
url.matchAgainst(genreSearchUrlPattern).doesMatch
}
val details = EnumMap<BookDetail, String>(BookDetail::class.java)
val tableParent: Element? = commonParent(status.key, origin.key)
val rows: List<Element>? = tableParent?.children()?.toList()
for (row in (rows ?: emptyList())) {
val cols = row.children() val cols = row.children()
val typeElement = cols.getOrNull(0) ?: continue val typeElement = cols.getOrNull(0) ?: continue
val valueElement = cols.getOrNull(1) ?: continue val valueElement = cols.getOrNull(1) ?: continue
@ -350,30 +447,37 @@ class DataExtractor(val extractionElement: Element) {
val typeText = typeElement.text() val typeText = typeElement.text()
val detail = BookDetail.from(typeText) ?: continue val detail = BookDetail.from(typeText) ?: continue
details[detail] = detail.elementProcessor(valueElement) details[detail] = detail.process(valueElement)
} }
details.getOrPut(BookDetail.AUTHOR) { authors.keys.joinToString(", ") { it.text() } } run {
details.getOrPut(BookDetail.ARTIST) { artists.keys.joinToString(", ") { it.text() } } val originGenre: String? = details[BookDetail.Origin]?.detailData?.let { originData ->
details.getOrPut(BookDetail.STATUS) { status.key.text() } when {
details.getOrPut(BookDetail.ORIGIN) { origin.key.text() } originData.matches(BookDetail.Origin.koreaRegex) -> "Manhwa"
originData.matches(BookDetail.Origin.chinaRegex) -> "Manhua"
details.getOrPut(BookDetail.GENRE) { genres.keys.joinToString(", ") { it.text() } } originData.matches(BookDetail.Origin.japanRegex) -> "Manga"
when (origin.value.queryParameter("origin")) {
"kr" -> "Manhwa"
"cn" -> "Manhua"
"jp" -> "Manga"
else -> null else -> null
}?.let { originGenre -> }
details[BookDetail.GENRE] = """${details[BookDetail.GENRE]}, $originGenre""" }
if (originGenre != null) {
details[BookDetail.Genre] = when (details.containsKey(BookDetail.Genre)) {
true -> {
val (label, data) = details[BookDetail.Genre]!!
BookDetail.ProcessedData(label, if (data.isBlank()) originGenre else """$data, $originGenre""")
}
false -> {
BookDetail.ProcessedData(BookDetail.Genre.label(null), originGenre)
}
}
}
} }
val title: Element? = extractionElement.selectFirst("h2[itemprop=title]") ?: extractionElement.selectFirst("h2") ?: run { val title: Element? = extractionElement.selectFirst("h2[itemprop=title]") ?: extractionElement.selectFirst("h2") ?: run {
// the common table is inside of a "row" wrapper that is the neighbour of the h2 containing the title // the common table is inside of a "row" wrapper that is the neighbour of the h2 containing the title
// if we sort of generalize this, the title should be the first // if we sort of generalize this, the title should be the first
// text-node-bearing child of the table's grandparent // text-node-bearing child of the table's grandparent
tableParent?.parent()?.parent()?.children()?.firstOrNull { it.textNodes().isNotEmpty() } detailsTable?.parent()?.parent()?.children()?.firstOrNull { it.textNodes().isNotEmpty() }
} }
val alerts: List<String> = extractionElement.select(".alert, .alert-info") val alerts: List<String> = extractionElement.select(".alert, .alert-info")
@ -415,11 +519,11 @@ class DataExtractor(val extractionElement: Element) {
PSBookDetails( PSBookDetails(
book = PSBook( book = PSBook(
bookThumbnailUrl(bookID, extension), bookThumbnailUrl(bookID, extension),
title?.text() ?: reportErrorToUser { "could not determine book title from details for $bookID" }, title?.text() ?: reportErrorToUser("DataExtractor.bookDetails") { "could not determine title for $bookID" },
url, url,
bookID, bookID,
), ),
detailsTable = details, details = details,
alertData = alerts, alertData = alerts,
description = description, description = description,
) )
@ -463,8 +567,8 @@ class DataExtractor(val extractionElement: Element) {
} }
companion object { companion object {
val all: Set<ChaptersTableColumnDataType> by lazy { setOf(Chapter, Group, Added, Language, Views) } val all: Set<ChaptersTableColumnDataType> by unexpectedErrorCatchingLazy { setOf(Chapter, Group, Added, Language, Views) }
val required: Set<ChaptersTableColumnDataType> by lazy { all.filterTo(LinkedHashSet()) { it.required } } val required: Set<ChaptersTableColumnDataType> by unexpectedErrorCatchingLazy { all.filterTo(LinkedHashSet()) { it.required } }
/** /**
* Takes the list of [headers] and returns a map that * Takes the list of [headers] and returns a map that
@ -517,7 +621,7 @@ class DataExtractor(val extractionElement: Element) {
* Then the `<tbody>` rows (`<tr>`) are one by one processed to find the ones that match the column (`<td>`) * Then the `<tbody>` rows (`<tr>`) are one by one processed to find the ones that match the column (`<td>`)
* size and data type positions that we care about. * size and data type positions that we care about.
*/ */
val bookChapters: Map<ScanGroup, List<BookChapter>> by lazy { val bookChapters: Map<ScanGroup, List<BookChapter>> by unexpectedErrorCatchingLazy {
data class RawTable(val self: Element, val thead: Element, val tbody: Element) data class RawTable(val self: Element, val thead: Element, val tbody: Element)
data class AnalyzedTable(val raw: RawTable, val columnDataTypes: Map<ChaptersTableColumnDataType, Int>, val dataRows: List<Elements>) data class AnalyzedTable(val raw: RawTable, val columnDataTypes: Map<ChaptersTableColumnDataType, Int>, val dataRows: List<Elements>)
@ -617,7 +721,7 @@ class DataExtractor(val extractionElement: Element) {
override fun compareTo(other: ChapterNumber): Int = comparator.compare(this, other) override fun compareTo(other: ChapterNumber): Int = comparator.compare(this, other)
companion object { companion object {
val comparator: Comparator<ChapterNumber> by lazy { compareBy({ it.main }, { it.sub }) } val comparator: Comparator<ChapterNumber> by unexpectedErrorCatchingLazy { compareBy({ it.main }, { it.sub }) }
val chapterNumberRegex: Regex = """(?:chapter|ch\.?)\s*(\d+)(?:\s*[.,-]\s*(\d+)?)?""".toRegex(RegexOption.IGNORE_CASE) val chapterNumberRegex: Regex = """(?:chapter|ch\.?)\s*(\d+)(?:\s*[.,-]\s*(\d+)?)?""".toRegex(RegexOption.IGNORE_CASE)
} }
} }

View File

@ -22,9 +22,7 @@ data class PathPattern(val paths: List<Regex?>) {
init { init {
if (paths.isEmpty()) { if (paths.isEmpty()) {
reportErrorToUser { reportErrorToUser { "Invalid PathPattern, cannot be empty!" }
"Invalid PathPattern, cannot be empty!"
}
} }
} }
} }
@ -51,9 +49,7 @@ data class PathMatchResult(val doesMatch: Boolean, val matchResults: List<MatchR
init { init {
if (matchResults?.isEmpty() == true) { if (matchResults?.isEmpty() == true) {
reportErrorToUser { reportErrorToUser { "Invalid PathMatchResult, matchResults must either be null or not empty!" }
"Invalid PathMatchResult, matchResults must either be null or not empty!"
}
} }
} }
} }

View File

@ -1,6 +1,11 @@
package eu.kanade.tachiyomi.extension.all.projectsuki package eu.kanade.tachiyomi.extension.all.projectsuki
import android.os.Build
import androidx.preference.PreferenceScreen import androidx.preference.PreferenceScreen
import eu.kanade.tachiyomi.extension.all.projectsuki.activities.INTENT_BOOK_QUERY_PREFIX
import eu.kanade.tachiyomi.extension.all.projectsuki.activities.INTENT_READ_QUERY_PREFIX
import eu.kanade.tachiyomi.extension.all.projectsuki.activities.INTENT_SEARCH_QUERY_PREFIX
import eu.kanade.tachiyomi.extension.all.projectsuki.activities.ProjectSukiSearchUrlActivity
import eu.kanade.tachiyomi.lib.randomua.getPrefCustomUA import eu.kanade.tachiyomi.lib.randomua.getPrefCustomUA
import eu.kanade.tachiyomi.lib.randomua.getPrefUAType import eu.kanade.tachiyomi.lib.randomua.getPrefUAType
import eu.kanade.tachiyomi.lib.randomua.setRandomUserAgent import eu.kanade.tachiyomi.lib.randomua.setRandomUserAgent
@ -143,9 +148,25 @@ internal val HttpUrl.rawRelative: String?
internal val reportPrefix: String internal val reportPrefix: String
get() = """Error! Report on GitHub (tachiyomiorg/tachiyomi-extensions)""" get() = """Error! Report on GitHub (tachiyomiorg/tachiyomi-extensions)"""
/** Just throw an [error], which will get caught by Tachiyomi: the message will be exposed as a [toast][android.widget.Toast]. */ /**
internal inline fun reportErrorToUser(message: () -> String): Nothing { * Simple named exception to differentiate it with all other "unexpected" exceptions.
error("""$reportPrefix: ${message()}""") * @see unexpectedErrorCatchingLazy
*/
internal class ProjectSukiException(message: String, cause: Throwable? = null) : Exception(message, cause)
/** Throws a [ProjectSukiException], which will get caught by Tachiyomi: the message will be exposed as a [toast][android.widget.Toast]. */
internal inline fun reportErrorToUser(locationHint: String? = null, message: () -> String): Nothing {
throw ProjectSukiException(
buildString {
append("[")
append(reportPrefix)
append("""]: """)
append(message())
if (!locationHint.isNullOrBlank()) {
append(" @$locationHint")
}
},
)
} }
/** Used when chapters don't have a [Language][DataExtractor.ChaptersTableColumnDataType.Language] column (if that ever happens). */ /** Used when chapters don't have a [Language][DataExtractor.ChaptersTableColumnDataType.Language] column (if that ever happens). */
@ -336,31 +357,94 @@ class ProjectSuki : HttpSource(), ConfigurableSource {
?.state ?.state
?.let { ProjectSukiFilters.SearchMode[it] } ?: ProjectSukiFilters.SearchMode.SMART ?.let { ProjectSukiFilters.SearchMode[it] } ?: ProjectSukiFilters.SearchMode.SMART
fun BookID.toMangasPageObservable(): Observable<MangasPage> {
val rawSManga = SManga.create().apply {
url = bookIDToURL().rawRelative ?: reportErrorToUser { "Could not create relative url for bookID: $this" }
}
return client.newCall(mangaDetailsRequest(rawSManga))
.asObservableSuccess()
.map { response -> mangaDetailsParse(response) }
.map { manga -> MangasPage(listOf(manga), hasNextPage = false) }
}
val queryAsURL: HttpUrl? by unexpectedErrorCatchingLazy { query.toHttpUrlOrNull() ?: """${homepageUri}$query""".toHttpUrlOrNull() }
val bookUrlMatch by unexpectedErrorCatchingLazy { queryAsURL?.matchAgainst(bookUrlPattern) }
val readUrlMatch by unexpectedErrorCatchingLazy { queryAsURL?.matchAgainst(chapterUrlPattern) }
return when { return when {
// sent by the url activity, might also be because the user entered a query via $ps: // sent by the url activity, might also be because the user entered a query via $ps-search:
// but that won't really happen unless the user wants to do that // but that won't really happen unless the user wants to do that
query.startsWith(INTENT_QUERY_PREFIX) -> { query.startsWith(INTENT_SEARCH_QUERY_PREFIX) -> {
val urlQuery = query.removePrefix(INTENT_QUERY_PREFIX) val urlQuery = query.removePrefix(INTENT_SEARCH_QUERY_PREFIX)
if (urlQuery.isBlank()) error("Empty search query!") if (urlQuery.isBlank()) error("Empty search query!")
val rawUrl = """${homepageUri.toASCIIString()}/search?$urlQuery""" val rawUrl = """${homepageUri.toASCIIString()}/search?$urlQuery"""
val url = rawUrl.toHttpUrlOrNull() ?: reportErrorToUser { val url = rawUrl.toHttpUrlOrNull() ?: reportErrorToUser { "Invalid search url: $rawUrl" }
"Invalid search url: $rawUrl"
}
client.newCall(GET(url, headers)) client.newCall(GET(url, headers))
.asObservableSuccess() .asObservableSuccess()
.map { response -> searchMangaParse(response, false) } .map { response -> searchMangaParse(response, overrideHasNextPage = false) }
}
// sent by the book activity
query.startsWith(INTENT_BOOK_QUERY_PREFIX) -> {
val bookid = query.removePrefix(INTENT_BOOK_QUERY_PREFIX)
if (bookid.isBlank()) error("Empty bookid!")
bookid.toMangasPageObservable()
}
// sent by the read activity
query.startsWith(INTENT_READ_QUERY_PREFIX) -> {
val bookid = query.removePrefix(INTENT_READ_QUERY_PREFIX)
if (bookid.isBlank()) error("Empty bookid!")
bookid.toMangasPageObservable()
}
bookUrlMatch?.doesMatch == true -> {
val bookid = bookUrlMatch!!["bookid"]!!.value
if (bookid.isBlank()) error("Empty bookid!")
bookid.toMangasPageObservable()
}
readUrlMatch?.doesMatch == true -> {
val bookid = readUrlMatch!!["bookid"]!!.value
if (bookid.isBlank()) error("Empty bookid!")
bookid.toMangasPageObservable()
} }
// use result from https://projectsuki.com/api/book/search // use result from https://projectsuki.com/api/book/search
searchMode == ProjectSukiFilters.SearchMode.SMART || searchMode == ProjectSukiFilters.SearchMode.SIMPLE -> { searchMode == ProjectSukiFilters.SearchMode.SMART -> {
val simpleMode = searchMode == ProjectSukiFilters.SearchMode.SIMPLE if (Build.VERSION.SDK_INT < Build.VERSION_CODES.N) {
error(
buildString {
append("Please enable ")
append(ProjectSukiFilters.SearchMode.SIMPLE)
append(" Search Mode: ")
append(ProjectSukiFilters.SearchMode.SMART)
append(" mode requires Android API version >= 24, but ")
append(Build.VERSION.SDK_INT)
append(" was found!")
},
)
}
client.newCall(ProjectSukiAPI.bookSearchRequest(json, headers)) client.newCall(ProjectSukiAPI.bookSearchRequest(json, headers))
.asObservableSuccess() .asObservableSuccess()
.map { response -> ProjectSukiAPI.parseBookSearchResponse(json, response) } .map { response -> ProjectSukiAPI.parseBookSearchResponse(json, response) }
.map { data -> data.toMangasPage(query, simpleMode) } .map { data -> SmartBookSearchHandler(query, data).mangasPage }
}
// use result from https://projectsuki.com/api/book/search
searchMode == ProjectSukiFilters.SearchMode.SIMPLE -> {
client.newCall(ProjectSukiAPI.bookSearchRequest(json, headers))
.asObservableSuccess()
.map { response -> ProjectSukiAPI.parseBookSearchResponse(json, response) }
.map { data -> data.simpleSearchMangasPage(query) }
} }
// use https://projectsuki.com/search // use https://projectsuki.com/search
@ -449,11 +533,11 @@ class ProjectSuki : HttpSource(), ConfigurableSource {
title = data.book.rawTitle title = data.book.rawTitle
thumbnail_url = data.book.thumbnail.toUri().toASCIIString() thumbnail_url = data.book.thumbnail.toUri().toASCIIString()
author = data.detailsTable[DataExtractor.BookDetail.AUTHOR] author = data.details[DataExtractor.BookDetail.Author]?.detailData
artist = data.detailsTable[DataExtractor.BookDetail.ARTIST] artist = data.details[DataExtractor.BookDetail.Artist]?.detailData
status = when (data.detailsTable[DataExtractor.BookDetail.STATUS]?.trim()?.lowercase(Locale.US)) { status = when (data.details[DataExtractor.BookDetail.Status]?.detailData?.trim()?.lowercase(Locale.US)) {
"ongoing" -> SManga.ONGOING "ongoing" -> SManga.ONGOING
"completed" -> SManga.PUBLISHING_FINISHED "completed" -> SManga.COMPLETED
"hiatus" -> SManga.ON_HIATUS "hiatus" -> SManga.ON_HIATUS
"cancelled" -> SManga.CANCELLED "cancelled" -> SManga.CANCELLED
else -> SManga.UNKNOWN else -> SManga.UNKNOWN
@ -461,7 +545,7 @@ class ProjectSuki : HttpSource(), ConfigurableSource {
description = buildString { description = buildString {
if (data.alertData.isNotEmpty()) { if (data.alertData.isNotEmpty()) {
appendLine("Alerts have been found, refreshing the manga later might help in removing them.") appendLine("Alerts have been found, refreshing the book/manga later might help in removing them.")
appendLine() appendLine()
data.alertData.forEach { data.alertData.forEach {
@ -469,14 +553,20 @@ class ProjectSuki : HttpSource(), ConfigurableSource {
appendLine() appendLine()
} }
appendLine(DESCRIPTION_DIVIDER)
appendLine()
appendLine() appendLine()
} }
appendLine(data.description) appendLine(data.description)
appendLine() appendLine()
data.detailsTable.forEach { (detail, value) -> appendLine(DESCRIPTION_DIVIDER)
append(detail.display) appendLine()
data.details.values.forEach { (label, value) ->
append(label)
append(" ") append(" ")
append(value.trim()) append(value.trim())
@ -485,11 +575,11 @@ class ProjectSuki : HttpSource(), ConfigurableSource {
} }
update_strategy = when (status) { update_strategy = when (status) {
SManga.CANCELLED, SManga.PUBLISHING_FINISHED -> UpdateStrategy.ONLY_FETCH_ONCE SManga.CANCELLED, SManga.COMPLETED, SManga.PUBLISHING_FINISHED -> UpdateStrategy.ONLY_FETCH_ONCE
else -> UpdateStrategy.ALWAYS_UPDATE else -> UpdateStrategy.ALWAYS_UPDATE
} }
genre = data.detailsTable[DataExtractor.BookDetail.GENRE]!! genre = data.details[DataExtractor.BookDetail.Genre]!!.detailData
} }
} }
@ -563,9 +653,7 @@ class ProjectSuki : HttpSource(), ConfigurableSource {
override fun fetchPageList(chapter: SChapter): Observable<List<Page>> { override fun fetchPageList(chapter: SChapter): Observable<List<Page>> {
val pathMatch: PathMatchResult = """${homepageUri.toASCIIString()}/${chapter.url}""".toHttpUrl().matchAgainst(chapterUrlPattern) val pathMatch: PathMatchResult = """${homepageUri.toASCIIString()}/${chapter.url}""".toHttpUrl().matchAgainst(chapterUrlPattern)
if (!pathMatch.doesMatch) { if (!pathMatch.doesMatch) {
reportErrorToUser { reportErrorToUser { "chapter url ${chapter.url} does not match expected pattern" }
"chapter url ${chapter.url} does not match expected pattern"
}
} }
return client.newCall(ProjectSukiAPI.chapterPagesRequest(json, headers, pathMatch["bookid"]!!.value, pathMatch["chapterid"]!!.value)) return client.newCall(ProjectSukiAPI.chapterPagesRequest(json, headers, pathMatch["bookid"]!!.value, pathMatch["chapterid"]!!.value))
@ -584,8 +672,12 @@ class ProjectSuki : HttpSource(), ConfigurableSource {
/** /**
* Not used in this extension, as we override [fetchPageList] to modify the default behaviour. * Not used in this extension, as we override [fetchPageList] to modify the default behaviour.
*/ */
override fun pageListParse(response: Response): List<Page> = reportErrorToUser { override fun pageListParse(response: Response): List<Page> = reportErrorToUser("ProjectSuki.pageListParse") {
// give a hint on who called this method // give a hint on who called this method
"invalid ${Thread.currentThread().stackTrace.take(3)}" "invalid ${Thread.currentThread().stackTrace.asSequence().drop(1).take(3).toList()}"
}
companion object {
private const val DESCRIPTION_DIVIDER: String = "/=/-/=/-/=/-/=/-/=/-/=/-/=/-/=/"
} }
} }

View File

@ -1,14 +1,8 @@
package eu.kanade.tachiyomi.extension.all.projectsuki package eu.kanade.tachiyomi.extension.all.projectsuki
import android.icu.text.BreakIterator
import android.icu.text.Collator
import android.icu.text.RuleBasedCollator
import android.icu.text.StringSearch
import android.os.Build
import eu.kanade.tachiyomi.network.POST import eu.kanade.tachiyomi.network.POST
import eu.kanade.tachiyomi.source.model.MangasPage import eu.kanade.tachiyomi.source.model.MangasPage
import eu.kanade.tachiyomi.source.model.Page import eu.kanade.tachiyomi.source.model.Page
import eu.kanade.tachiyomi.source.model.SManga
import kotlinx.serialization.SerialName import kotlinx.serialization.SerialName
import kotlinx.serialization.Serializable import kotlinx.serialization.Serializable
import kotlinx.serialization.encodeToString import kotlinx.serialization.encodeToString
@ -26,7 +20,6 @@ import okhttp3.RequestBody.Companion.toRequestBody
import okhttp3.Response import okhttp3.Response
import org.jsoup.Jsoup import org.jsoup.Jsoup
import org.jsoup.nodes.Element import org.jsoup.nodes.Element
import java.text.StringCharacterIterator
/** /**
* @see EXTENSION_INFO Found in ProjectSuki.kt * @see EXTENSION_INFO Found in ProjectSuki.kt
@ -95,9 +88,7 @@ object ProjectSukiAPI {
) { ) {
init { init {
if (first != "true" && first != "false") { if (first != "true" && first != "false") {
reportErrorToUser { reportErrorToUser { "PagesRequestData, first was \"$first\"" }
"PagesRequestData, first was \"$first\""
}
} }
} }
} }
@ -136,9 +127,7 @@ object ProjectSukiAPI {
?.tryAs<JsonObject>() ?.tryAs<JsonObject>()
?.get("src") ?.get("src")
?.tryAs<JsonPrimitive>() ?.tryAs<JsonPrimitive>()
?.content ?: reportErrorToUser { ?.content ?: reportErrorToUser { "chapter pages aren't in the expected format!" }
"chapter pages aren't in the expected format!"
}
// we can handle relative urls by specifying manually the location of the "document" // we can handle relative urls by specifying manually the location of the "document"
val srcFragment: Element = Jsoup.parseBodyFragment(rawSrc, homepageUri.toASCIIString()) val srcFragment: Element = Jsoup.parseBodyFragment(rawSrc, homepageUri.toASCIIString())
@ -150,9 +139,7 @@ object ProjectSukiAPI {
.filterValues { it.doesMatch } // make sure they are the urls we expect .filterValues { it.doesMatch } // make sure they are the urls we expect
if (urls.isEmpty()) { if (urls.isEmpty()) {
reportErrorToUser { reportErrorToUser { "chapter pages URLs aren't in the expected format!" }
"chapter pages URLs aren't in the expected format!"
}
} }
return urls.entries return urls.entries
@ -197,18 +184,14 @@ object ProjectSukiAPI {
.getOrNull() .getOrNull()
?.tryAs<JsonObject>() ?.tryAs<JsonObject>()
?.get("data") ?.get("data")
?.tryAs<JsonObject>() ?: reportErrorToUser { ?.tryAs<JsonObject>() ?: reportErrorToUser { "books data isn't in the expected format!" }
"books data isn't in the expected format!"
}
val refined: Map<BookID, BookTitle> = buildMap { val refined: Map<BookID, BookTitle> = buildMap {
data.forEach { (id: BookID, valueObj: JsonElement) -> data.forEach { (id: BookID, valueObj: JsonElement) ->
val title: BookTitle = valueObj.tryAs<JsonObject>() val title: BookTitle = valueObj.tryAs<JsonObject>()
?.get("value") ?.get("value")
?.tryAs<JsonPrimitive>() ?.tryAs<JsonPrimitive>()
?.content ?: reportErrorToUser { ?.content ?: reportErrorToUser { "books data isn't in the expected format!" }
"books data isn't in the expected format!"
}
this[id] = title this[id] = title
} }
@ -226,17 +209,9 @@ private val alphaNumericRegex = """\p{Alnum}+""".toRegex(RegexOption.IGNORE_CASE
* If Even a single "word" from [searchQuery] matches, then the manga will be included, * If Even a single "word" from [searchQuery] matches, then the manga will be included,
* but sorting is done based on the amount of matches. * but sorting is done based on the amount of matches.
*/ */
internal fun Map<BookID, BookTitle>.toMangasPage(searchQuery: String, useSimpleMode: Boolean): MangasPage { internal fun Map<BookID, BookTitle>.simpleSearchMangasPage(searchQuery: String): MangasPage {
data class Match(val bookID: BookID, val title: BookTitle, val count: Int) { data class Match(val bookID: BookID, val title: BookTitle, val count: Int)
val bookUrl: HttpUrl = homepageUrl.newBuilder()
.addPathSegment("book")
.addPathSegment(bookID)
.build()
}
when {
useSimpleMode -> {
// simple search, possibly faster
val words: Set<String> = alphaNumericRegex.findAll(searchQuery).mapTo(HashSet()) { it.value } val words: Set<String> = alphaNumericRegex.findAll(searchQuery).mapTo(HashSet()) { it.value }
val matches: Map<BookID, Match> = mapValues { (bookID, bookTitle) -> val matches: Map<BookID, Match> = mapValues { (bookID, bookTitle) ->
@ -258,96 +233,8 @@ internal fun Map<BookID, BookTitle>.toMangasPage(searchQuery: String, useSimpleM
Match(bookID, bookTitle, matchesCount) Match(bookID, bookTitle, matchesCount)
}.filterValues { it.count > 0 } }.filterValues { it.count > 0 }
return MangasPage( return matches.entries
mangas = matches.entries
.sortedWith(compareBy({ -it.value.count }, { it.value.title })) .sortedWith(compareBy({ -it.value.count }, { it.value.title }))
.map { (bookID, match: Match) -> .associate { (bookID, match) -> bookID to match.title }
SManga.create().apply { .toMangasPage()
title = match.title
url = match.bookUrl.rawRelative ?: reportErrorToUser { "Could not relativize ${match.bookUrl}" }
thumbnail_url = bookThumbnailUrl(bookID, "").toUri().toASCIIString()
}
},
hasNextPage = false,
)
}
Build.VERSION.SDK_INT >= Build.VERSION_CODES.N -> {
// use ICU, better
val searchWords: Set<String> = BreakIterator.getWordInstance().run {
text = StringCharacterIterator(searchQuery)
var left = first()
var right = next()
buildSet {
while (right != BreakIterator.DONE) {
if (ruleStatus != BreakIterator.WORD_NONE) {
add(searchQuery.substring(left, right))
}
left = right
right = next()
}
}
}
val stringSearch = StringSearch(
/* pattern = */ "dummy",
/* target = */ StringCharacterIterator("dummy"),
/* collator = */
(Collator.getInstance() as RuleBasedCollator).apply {
isCaseLevel = true
strength = Collator.PRIMARY
decomposition = Collator.CANONICAL_DECOMPOSITION
},
)
val matches: Map<BookID, Match> = mapValues { (bookID, bookTitle) ->
stringSearch.target = StringCharacterIterator(bookTitle)
val matchesCount: Int = searchWords.sumOf { word ->
val search: StringSearch = stringSearch.apply {
this.pattern = word
}
var count = 0
var idx = search.first()
while (idx != StringSearch.DONE) {
count++
idx = search.next()
}
count
}
Match(bookID, bookTitle, matchesCount)
}.filterValues { it.count > 0 }
return MangasPage(
mangas = matches.entries
.sortedWith(compareBy({ -it.value.count }, { it.value.title }))
.map { (bookID, match: Match) ->
SManga.create().apply {
title = match.title
url = match.bookUrl.rawRelative ?: reportErrorToUser { "Could not relativize ${match.bookUrl}" }
thumbnail_url = bookThumbnailUrl(bookID, "").toUri().toASCIIString()
}
},
hasNextPage = false,
)
}
else -> error(
buildString {
append("Please enable ")
append(ProjectSukiFilters.SearchMode.SIMPLE)
append(" Search Mode: ")
append(ProjectSukiFilters.SearchMode.SMART)
append(" search requires Android API version >= 24, but ")
append(Build.VERSION.SDK_INT)
append(" was found!")
},
)
}
} }

View File

@ -0,0 +1,223 @@
package eu.kanade.tachiyomi.extension.all.projectsuki
import android.icu.text.BreakIterator
import android.icu.text.Collator
import android.icu.text.Normalizer2
import android.icu.text.RuleBasedCollator
import android.icu.text.StringSearch
import android.os.Build
import androidx.annotation.RequiresApi
import eu.kanade.tachiyomi.source.model.MangasPage
import eu.kanade.tachiyomi.source.model.SManga
import okhttp3.HttpUrl
import java.text.StringCharacterIterator
import java.util.TreeMap
import kotlin.properties.PropertyDelegateProvider
import kotlin.reflect.KProperty
/**
* @see EXTENSION_INFO Found in ProjectSuki.kt
*/
@Suppress("unused")
private inline val INFO: Nothing get() = error("INFO")
private typealias IsWord = Boolean
/**
* Handles the "Smart" book search mode, requires [API][RequiresApi] [Build.VERSION_CODES.N]
* because it uses [android.icu.text] classes.
*
* Tries to avoid wasting resource by normalizing text only once and using [ThreadLocal] instances.
*/
@Suppress("MemberVisibilityCanBePrivate")
@RequiresApi(Build.VERSION_CODES.N)
class SmartBookSearchHandler(val rawQuery: String, val rawBooksData: Map<BookID, BookTitle>) {
data class WordsData(val words: List<String>, val extra: List<String>, val wordRanges: List<IntRange>)
data class CollatedElement<Cat>(val value: String, val range: IntRange, val origin: String, val category: Cat)
@Suppress("NOTHING_TO_INLINE")
private inline fun BreakIterator.collate(text: String): List<CollatedElement<Unit>> = collate(text) {}
private inline fun <Cat> BreakIterator.collate(text: String, categorizer: (ruleStatus: Int) -> Cat): List<CollatedElement<Cat>> {
this.text = StringCharacterIterator(text)
var left = first()
var right = next()
return buildList {
while (right != BreakIterator.DONE) {
val cat = categorizer(ruleStatus)
val value = text.substring(left, right)
add(CollatedElement(value, left..right, text, cat))
left = right
right = next()
}
}
}
private val normQuery by unexpectedErrorCatchingLazy { normalizer.normalize(rawQuery) }
val wordsData: WordsData by unexpectedErrorCatchingLazy {
val charBreak = charBreak
val wordBreak = wordBreak
val words: List<CollatedElement<IsWord>> = wordBreak.collate(normQuery) { ruleStatus ->
when (ruleStatus) {
BreakIterator.WORD_NONE -> false
else -> true
}
}
val extra: MutableList<String> = ArrayList()
words.forEach { collatedElement ->
if (!collatedElement.category) {
// not a "word" per-say
extra.addAll(charBreak.collate(collatedElement.value) { Unit }.map { it.value })
}
}
WordsData(
words = words.filter { it.category }.map { it.value },
extra = extra,
wordRanges = words.filter { it.category }.map { it.range },
)
}
val normalizedBooksData: Map<BookID, BookTitle> by unexpectedErrorCatchingLazy {
val normalizer = normalizer
rawBooksData.mapValues { normalizer.normalize(it.value) }
}
@OptIn(ExperimentalUnsignedTypes::class)
fun wordScoreFor(matchedText: String): UInt {
// not starting from index 0 is intentional here
return fib32.getOrElse(charBreak.collate(matchedText).size) { fib32.last() }
}
val filteredBooks: Set<BookID> by unexpectedErrorCatchingLazy {
val stringSearch = stringSearch
val wordsData = wordsData
val booksData: Map<BookID, BookTitle> = normalizedBooksData
class Counter(var value: UInt)
val scored: Map<BookID, Counter> = booksData.mapValues { Counter(0u) }
booksData.forEach { (bookID, normTitle) ->
val score = scored[bookID]!!
stringSearch.target = StringCharacterIterator(normTitle)
wordsData.words.forEach { word ->
stringSearch.pattern = word
var idx = stringSearch.first()
while (idx != StringSearch.DONE) {
score.value += wordScoreFor(stringSearch.matchedText)
idx = stringSearch.next()
}
}
wordsData.extra.forEach { extra ->
stringSearch.pattern = extra
var idx = stringSearch.first()
while (idx != StringSearch.DONE) {
score.value += SCORE_EXTRA
idx = stringSearch.next()
}
}
}
val byScore: TreeMap<UInt, MutableList<Map.Entry<BookID, Counter>>> = scored.entries.groupByTo(TreeMap(reverseOrder<UInt>())) { it.value.value }
val highest = byScore.firstKey().toFloat()
val included: MutableSet<BookID> = LinkedHashSet()
for ((score, group) in byScore) {
val include = score > 0u && (included.size < MINIMUM_RESULTS || (score.toFloat() / highest) >= NEEDED_FRACTIONAL_SCORE_FOR_INCLUSION)
if (!include) break
included.addAll(group.map { it.key })
}
included
}
val mangasPage: MangasPage by unexpectedErrorCatchingLazy {
filteredBooks.associateWith { rawBooksData[it]!! }
.toMangasPage()
}
companion object {
private inline fun <T> threadLocal(crossinline initializer: () -> T) = PropertyDelegateProvider<Any?, ThreadLocal<T>> { _: Any?, _: KProperty<*> ->
object : ThreadLocal<T>() {
override fun initialValue(): T? = initializer()
}
}
private operator fun <T> ThreadLocal<T>.getValue(thisRef: Any?, property: KProperty<*>): T {
return get() ?: reportErrorToUser("SmartBookSearchHandler.${property.name}") { "null initialValue" }
}
private val charBreak: BreakIterator by threadLocal { BreakIterator.getCharacterInstance() }
private val wordBreak: BreakIterator by threadLocal { BreakIterator.getWordInstance() }
private val normalizer: Normalizer2 by threadLocal { Normalizer2.getNFKCCasefoldInstance() }
private val collator: RuleBasedCollator by threadLocal {
(Collator.getInstance() as RuleBasedCollator).apply {
isCaseLevel = true
strength = Collator.PRIMARY
decomposition = Collator.NO_DECOMPOSITION // !! must handle this ourselves !!
}
}
private val stringSearch: StringSearch by threadLocal {
StringSearch(
/* pattern = */ "dummy",
/* target = */ StringCharacterIterator("dummy"),
/* collator = */ collator,
).apply {
isOverlapping = true
}
}
/** first 32 fibonacci numbers */
@JvmStatic
@OptIn(ExperimentalUnsignedTypes::class)
private val fib32: UIntArray = uintArrayOf(
1u, 1u, 2u, 3u,
5u, 8u, 13u, 21u,
34u, 55u, 89u, 144u,
233u, 377u, 610u, 987u,
1597u, 2584u, 4181u, 6765u,
10946u, 17711u, 28657u, 46368u,
75025u, 121393u, 196418u, 317811u,
514229u, 832040u, 1346269u, 2178309u,
)
private const val SCORE_EXTRA: UInt = 1u
private const val NEEDED_FRACTIONAL_SCORE_FOR_INCLUSION: Float = 0.5f
private const val MINIMUM_RESULTS: Int = 8
}
}
/** simply creates an https://projectsuki.com/book/<bookid> [HttpUrl] */
internal fun BookID.bookIDToURL(): HttpUrl {
return homepageUrl.newBuilder()
.addPathSegment("book")
.addPathSegment(this)
.build()
}
internal fun Map<BookID, BookTitle>.toMangasPage(hasNextPage: Boolean = false): MangasPage = entries.toMangasPage(hasNextPage)
internal fun Iterable<Map.Entry<BookID, BookTitle>>.toMangasPage(hasNextPage: Boolean = false): MangasPage {
return MangasPage(
mangas = map { (bookID: BookID, bookTitle: BookTitle) ->
SManga.create().apply {
title = bookTitle
url = bookID.bookIDToURL().rawRelative ?: reportErrorToUser { "Could not create relative url for bookID: $bookID" }
thumbnail_url = bookThumbnailUrl(bookID, "").toUri().toASCIIString()
}
},
hasNextPage = hasNextPage,
)
}

View File

@ -0,0 +1,74 @@
package eu.kanade.tachiyomi.extension.all.projectsuki.activities
import android.app.Activity
import android.content.ActivityNotFoundException
import android.content.Intent
import android.os.Bundle
import android.util.Log
import eu.kanade.tachiyomi.extension.all.projectsuki.EXTENSION_INFO
import eu.kanade.tachiyomi.extension.all.projectsuki.SHORT_FORM_ID
import kotlin.system.exitProcess
/**
* @see EXTENSION_INFO Found in ProjectSuki.kt
*/
@Suppress("unused")
private inline val INFO: Nothing get() = error("INFO")
internal const val INTENT_BOOK_QUERY_PREFIX: String = """${'$'}$SHORT_FORM_ID-book:"""
/**
* See [handleIntentAction](https://github.com/tachiyomiorg/tachiyomi/blob/0f9895eec8f5808210f291d1e0ef5cc9f73ccb44/app/src/main/java/eu/kanade/tachiyomi/ui/main/MainActivity.kt#L401)
* and [GlobalSearchScreen](https://github.com/tachiyomiorg/tachiyomi/blob/0f9895eec8f5808210f291d1e0ef5cc9f73ccb44/app/src/main/java/eu/kanade/tachiyomi/ui/browse/source/globalsearch/GlobalSearchScreen.kt#L19)
* (these are permalinks, search for updated variants).
*
* See [AndroidManifest.xml](https://developer.android.com/guide/topics/manifest/manifest-intro)
* for what URIs this [Activity](https://developer.android.com/guide/components/activities/intro-activities)
* can receive.
*
* For this specific class you can test the activity by doing (see [CONTRIBUTING](https://github.com/tachiyomiorg/tachiyomi-extensions/blob/master/CONTRIBUTING.md#url-intent-filter)):
* ```
* adb shell am start -d "https://projectsuki.com/book/192047" -a android.intent.action.VIEW
* ```
*
* If multiple devices are present, [see this answer](https://stackoverflow.com/a/14655015).
*
* Note that Tachiyomi extension's UrlActivities [do not have access to the Kotlin environment](https://github.com/tachiyomiorg/tachiyomi-extensions/pull/19323#issuecomment-1858932113)
* (specifically Intrinsics).
* To avoid using Java, multiple classes have been provided for different URLs.
*
* @author Federico d'Alonzo &lt;me@npgx.dev&gt;
*/
class ProjectSukiBookUrlActivity : Activity() {
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
if ((intent?.data?.pathSegments?.size ?: 0) >= 2) {
// Project Suki Url Activity Book
Log.e("PSUABook", "could not handle URI ${intent?.data} from intent $intent")
}
val intent = Intent().apply {
// tell tachiyomi we want to search for something
action = "eu.kanade.tachiyomi.SEARCH"
// "filter" for our own extension instead of doing a global search
putExtra("filter", packageName)
// value that will be passed onto the "query" parameter in fetchSearchManga
putExtra("query", "$INTENT_BOOK_QUERY_PREFIX${intent?.data?.pathSegments?.get(1)}")
}
try {
// actually do the thing
startActivity(intent)
} catch (e: ActivityNotFoundException) {
// tachiyomi isn't installed (?)
// Project Suki Url Activity Book
Log.e("PSUABook", e.toString())
}
// we're done
finish()
// just for safety
exitProcess(0)
}
}

View File

@ -0,0 +1,74 @@
package eu.kanade.tachiyomi.extension.all.projectsuki.activities
import android.app.Activity
import android.content.ActivityNotFoundException
import android.content.Intent
import android.os.Bundle
import android.util.Log
import eu.kanade.tachiyomi.extension.all.projectsuki.EXTENSION_INFO
import eu.kanade.tachiyomi.extension.all.projectsuki.SHORT_FORM_ID
import kotlin.system.exitProcess
/**
* @see EXTENSION_INFO Found in ProjectSuki.kt
*/
@Suppress("unused")
private inline val INFO: Nothing get() = error("INFO")
internal const val INTENT_READ_QUERY_PREFIX: String = """${'$'}$SHORT_FORM_ID-read:"""
/**
* See [handleIntentAction](https://github.com/tachiyomiorg/tachiyomi/blob/0f9895eec8f5808210f291d1e0ef5cc9f73ccb44/app/src/main/java/eu/kanade/tachiyomi/ui/main/MainActivity.kt#L401)
* and [GlobalSearchScreen](https://github.com/tachiyomiorg/tachiyomi/blob/0f9895eec8f5808210f291d1e0ef5cc9f73ccb44/app/src/main/java/eu/kanade/tachiyomi/ui/browse/source/globalsearch/GlobalSearchScreen.kt#L19)
* (these are permalinks, search for updated variants).
*
* See [AndroidManifest.xml](https://developer.android.com/guide/topics/manifest/manifest-intro)
* for what URIs this [Activity](https://developer.android.com/guide/components/activities/intro-activities)
* can receive.
*
* For this specific class you can test the activity by doing (see [CONTRIBUTING](https://github.com/tachiyomiorg/tachiyomi-extensions/blob/master/CONTRIBUTING.md#url-intent-filter)):
* ```
* adb shell am start -d "https://projectsuki.com/read/192047/5069/1" -a android.intent.action.VIEW
* ```
*
* If multiple devices are present, [see this answer](https://stackoverflow.com/a/14655015).
*
* Note that Tachiyomi extension's UrlActivities [do not have access to the Kotlin environment](https://github.com/tachiyomiorg/tachiyomi-extensions/pull/19323#issuecomment-1858932113)
* (specifically Intrinsics).
* To avoid using Java, multiple classes have been provided for different URLs.
*
* @author Federico d'Alonzo &lt;me@npgx.dev&gt;
*/
class ProjectSukiReadUrlActivity : Activity() {
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
if ((intent?.data?.pathSegments?.size ?: 0) >= 2) {
// Project Suki Url Activity Read
Log.e("PSUARead", "could not handle URI ${intent?.data} from intent $intent")
}
val intent = Intent().apply {
// tell tachiyomi we want to search for something
action = "eu.kanade.tachiyomi.SEARCH"
// "filter" for our own extension instead of doing a global search
putExtra("filter", packageName)
// value that will be passed onto the "query" parameter in fetchSearchManga
putExtra("query", "$INTENT_READ_QUERY_PREFIX${intent?.data?.pathSegments?.get(1)}")
}
try {
// actually do the thing
startActivity(intent)
} catch (e: ActivityNotFoundException) {
// tachiyomi isn't installed (?)
// Project Suki Url Activity Read
Log.e("PSUARead", e.toString())
}
// we're done
finish()
// just for safety
exitProcess(0)
}
}

View File

@ -1,10 +1,12 @@
package eu.kanade.tachiyomi.extension.all.projectsuki package eu.kanade.tachiyomi.extension.all.projectsuki.activities
import android.app.Activity import android.app.Activity
import android.content.ActivityNotFoundException import android.content.ActivityNotFoundException
import android.content.Intent import android.content.Intent
import android.os.Bundle import android.os.Bundle
import android.util.Log import android.util.Log
import eu.kanade.tachiyomi.extension.all.projectsuki.EXTENSION_INFO
import eu.kanade.tachiyomi.extension.all.projectsuki.SHORT_FORM_ID
import kotlin.system.exitProcess import kotlin.system.exitProcess
/** /**
@ -13,10 +15,7 @@ import kotlin.system.exitProcess
@Suppress("unused") @Suppress("unused")
private inline val INFO: Nothing get() = error("INFO") private inline val INFO: Nothing get() = error("INFO")
/** internal const val INTENT_SEARCH_QUERY_PREFIX: String = """${'$'}$SHORT_FORM_ID-search:"""
* `$ps:`
*/
internal const val INTENT_QUERY_PREFIX: String = """${'$'}$SHORT_FORM_ID:"""
/** /**
* See [handleIntentAction](https://github.com/tachiyomiorg/tachiyomi/blob/0f9895eec8f5808210f291d1e0ef5cc9f73ccb44/app/src/main/java/eu/kanade/tachiyomi/ui/main/MainActivity.kt#L401) * See [handleIntentAction](https://github.com/tachiyomiorg/tachiyomi/blob/0f9895eec8f5808210f291d1e0ef5cc9f73ccb44/app/src/main/java/eu/kanade/tachiyomi/ui/main/MainActivity.kt#L401)
@ -32,6 +31,12 @@ internal const val INTENT_QUERY_PREFIX: String = """${'$'}$SHORT_FORM_ID:"""
* adb shell am start -d "https://projectsuki.com/search?q=omniscient" -a android.intent.action.VIEW * adb shell am start -d "https://projectsuki.com/search?q=omniscient" -a android.intent.action.VIEW
* ``` * ```
* *
* If multiple devices are present, [see this answer](https://stackoverflow.com/a/14655015)
*
* Note that Tachiyomi extension's UrlActivities [do not have access to the Kotlin environment](https://github.com/tachiyomiorg/tachiyomi-extensions/pull/19323#issuecomment-1858932113)
* (specifically Intrinsics).
* To avoid using Java, multiple classes have been provided for different URLs.
*
* @author Federico d'Alonzo &lt;me@npgx.dev&gt; * @author Federico d'Alonzo &lt;me@npgx.dev&gt;
*/ */
class ProjectSukiSearchUrlActivity : Activity() { class ProjectSukiSearchUrlActivity : Activity() {
@ -39,7 +44,8 @@ class ProjectSukiSearchUrlActivity : Activity() {
super.onCreate(savedInstanceState) super.onCreate(savedInstanceState)
if (intent?.data?.pathSegments?.size != 1) { if (intent?.data?.pathSegments?.size != 1) {
Log.e("PSUrlActivity", "could not handle URI ${intent?.data} from intent $intent") // Project Suki Url Activity Search
Log.e("PSUASearch", "could not handle URI ${intent?.data} from intent $intent")
} }
val intent = Intent().apply { val intent = Intent().apply {
@ -48,7 +54,7 @@ class ProjectSukiSearchUrlActivity : Activity() {
// "filter" for our own extension instead of doing a global search // "filter" for our own extension instead of doing a global search
putExtra("filter", packageName) putExtra("filter", packageName)
// value that will be passed onto the "query" parameter in fetchSearchManga // value that will be passed onto the "query" parameter in fetchSearchManga
putExtra("query", "${INTENT_QUERY_PREFIX}${intent?.data?.query}") putExtra("query", "$INTENT_SEARCH_QUERY_PREFIX${intent?.data?.query}")
} }
try { try {
@ -56,7 +62,8 @@ class ProjectSukiSearchUrlActivity : Activity() {
startActivity(intent) startActivity(intent)
} catch (e: ActivityNotFoundException) { } catch (e: ActivityNotFoundException) {
// tachiyomi isn't installed (?) // tachiyomi isn't installed (?)
Log.e("PSUrlActivity", e.toString()) // Project Suki Url Activity Search
Log.e("PSUASearch", e.toString())
} }
// we're done // we're done