107 Commits

Author SHA1 Message Date
b610ec734e Chapter readonly struct 2023-07-30 17:09:39 +02:00
abf587377c API: Changed uninstantiated class Program to static 2023-07-30 17:09:30 +02:00
437349bd27 TrangaSettings changed set directive 2023-07-30 17:09:10 +02:00
000539d6a6 Moved logger to Trangasettings 2023-07-30 17:08:43 +02:00
b4bef25a22 Moved downloadclient to separate file 2023-07-30 17:04:43 +02:00
579e400a5d Moved class to appropriate namespaces 2023-07-30 17:01:54 +02:00
8af2b12fc0 Moved class to appropriate namespaces 2023-07-30 16:26:29 +02:00
bad4330330 introduce branch cuttingedge 2023-07-30 16:21:04 +02:00
42596752d3 FIX: null Publications in tasks 2023-07-29 18:55:06 +02:00
16238c590b Remove UpdateLibrariesTask 2023-07-29 18:20:41 +02:00
9f38dc3b6a Revert "Remove UpdateLibrariesTask"
This reverts commit de14ff0b75.
2023-07-29 18:18:02 +02:00
485637d99a Added Min-Chapter-Number to API 2023-07-28 10:47:36 +02:00
de14ff0b75 Remove UpdateLibrariesTask 2023-07-28 10:41:20 +02:00
f947c37bd6 Change website context to revert location to / instead of /Website 2023-07-28 10:30:54 +02:00
77eec0f696 Fix wrong deserialization 2023-07-21 00:32:18 +02:00
18323f9f51 remove debug 2023-07-21 00:22:41 +02:00
2cd2b6842d arch armv7 fails to build 2023-07-21 00:20:13 +02:00
09f815903f arch arm64 fails to build 2023-07-21 00:18:08 +02:00
c108478039 context 2 2023-07-21 00:15:35 +02:00
74289e43b7 context 2023-07-21 00:14:07 +02:00
2779f9ba09 Merge remote-tracking branch 'origin/master' 2023-07-21 00:12:23 +02:00
59a8e556f0 wrong build path 2023-07-21 00:12:09 +02:00
074b137b5c Merge pull request 'dev' (!48) from dev into master
Reviewed-on: #48
2023-07-21 00:10:33 +02:00
3cb2540794 debugging 2023-07-21 00:09:59 +02:00
02c9934896 change context back to API 2023-07-21 00:09:51 +02:00
b2e1c95bca Merge remote-tracking branch 'origin/master' 2023-07-21 00:07:44 +02:00
8c9e3ea6b6 Merge pull request 'split into two actions, dont always build tranga-base' (!47) from dev into master
Reviewed-on: #47
2023-07-21 00:07:16 +02:00
db441607ad Merge branch 'master' into dev 2023-07-21 00:04:24 +02:00
91c56783dc restore absolute path 2023-07-21 00:03:32 +02:00
2c288eeeea Don't rebuild tranga-base every time. 2023-07-20 23:54:30 +02:00
57a1ea91fc Merge pull request 'dev' (!46) from dev into master
Reviewed-on: #46
2023-07-20 23:50:15 +02:00
06138a3927 Workflow change context 2023-07-20 23:49:33 +02:00
84b053e672 Merge remote-tracking branch 'origin/dev' 2023-07-20 23:44:17 +02:00
0fe0cbc4ad Merge pull request #34 from C9Glax/dev
Unsupported arch
2023-07-20 23:42:04 +02:00
62e6ce8363 remove unsupported platforms 2023-07-20 23:38:10 +02:00
a4f3ec6580 Merge pull request #33 from schklom/master
Automatic build of Docker images for many platforms (ARM too)
2023-07-20 23:25:10 +02:00
8b4e996b7e Create dependabot.yml 2023-07-20 23:10:46 +02:00
964540d30f Create docker-image.yml 2023-07-20 23:10:15 +02:00
fa69f4488f Removed UpdateLibraryTask (deprecated).
Libraries will be updated on new Chapters downloaded.
Added Migrator, for future file-changes
2023-07-20 18:15:14 +02:00
42c2876188 Mangakatana chapter num fix 2023-07-16 20:22:33 +02:00
715244ff1b Mangasee more logging 2023-07-16 18:15:28 +02:00
2333cd9095 Mangasee more bad words 2023-07-16 18:15:11 +02:00
c8225db4fe #30 #31 2023-07-16 17:47:00 +02:00
6741ca096b Startup Message 2023-07-16 17:38:42 +02:00
a897a7b3a2 Better Logger.
Includes a formatted Console-Log
2023-07-16 17:33:15 +02:00
0f8932e712 Fixed missing logger for notificationManagers on deserialization 2023-07-09 21:38:49 +02:00
78023ef0fd resolves #21 lunasea 2023-07-09 21:35:15 +02:00
d171f34e4e Update README.md 2023-07-07 14:23:33 +02:00
aa0dc4fa35 Fixes single result redirect 2023-07-06 02:09:56 +02:00
25f48592c0 Added more badwords to filter out when searching mangasee, resolves #26 2023-07-04 22:44:01 +02:00
398ac304d2 Update Komga/Kavita immediately after new chapter is Downloaded 2023-07-03 00:01:08 +02:00
58a62f8272 Mangasee search all title-fields. 2023-07-02 23:54:02 +02:00
86752c9a7e Order of task Execution by due-time 2023-07-02 23:10:16 +02:00
f9a7828d02 Moved notification back to DownloadChapterTask
temp: Dont wait for childTasks to finish to finish parent task
2023-07-02 23:06:24 +02:00
c97ff69148 Fix for new publications: Add to collection 2023-07-02 22:46:01 +02:00
1735bbcf8a Fix wrong query from allTasks to runningTasks 2023-06-30 00:23:00 +02:00
9ae8ca65df resolves #25 characters encoding mistake 2023-06-29 21:09:42 +02:00
00599cd24e Infinite loop on unavailable chapters 2023-06-28 23:00:24 +02:00
6d5618a1f7 Infinite loop on unavailable chapters 2023-06-28 22:46:22 +02:00
a1202a875d Moved sucessstate to taskmanager 2023-06-28 22:43:46 +02:00
98946b4aa3 Fixed null chapterNumber on mangadex 2023-06-28 22:43:24 +02:00
41b6bb77b6 Moved GetPublicationsFromConnector to connector.
Moved GetNewChaptersList to Connector.
Removed knownPublications file
Renamed chapterCollection to collection and only contains Publications
2023-06-28 22:43:03 +02:00
e70a14ca56 Only send notifications if more than 0 new chapters 2023-06-28 19:23:06 +02:00
b099da1156 Chapter fix RegexMatching on chapter number 2023-06-28 00:13:23 +02:00
01d1f922c2 MangaDex chapterNumber non.nullable 2023-06-28 00:13:09 +02:00
47a80d67a8 TrangaTask Success-State and child task deletion 2023-06-27 23:55:13 +02:00
16e3549455 Export Data after deleting task 2023-06-27 23:54:44 +02:00
be8c6b50ba Notification moved to TrangaTask 2023-06-27 23:37:13 +02:00
a38fcf50ca nullable types removed 2023-06-27 23:25:35 +02:00
82f6c7b3fe Moved GetArchiveFilePath, CheckChapterIsDownloaded and GetComicInfoXmlString to Chapter.cs 2023-06-27 23:22:23 +02:00
5586d2c104 Connector CheckChapterIsDownloaded more Regex 2023-06-27 23:14:22 +02:00
62dc9fee2a GetComicInfoXmlString: protected -> internal 2023-06-27 23:09:09 +02:00
ac96fca6dc Chapter illegalstring regex 2023-06-27 23:08:29 +02:00
25a6ceff10 Remove sortNumber-field from Chapter
API: Change Tasks/Progress chapterSortNumber to ChapterNumber
2023-06-27 23:06:37 +02:00
b3e1d39d0f Rename Connector.SearchChapters -> SelectChapters
Added "a(ll)"-option to SelectChapters
2023-06-27 23:02:55 +02:00
2833b7f22a Remove Legacy support for "DownloadNewChapters" 2023-06-27 22:59:33 +02:00
cbdd305b69 TaskManager AddTask make better use of GetTasksMatching and GetTasksMatching easier usage 2023-06-27 22:59:23 +02:00
b88890817e TaskManager _runningDownloadChapterTasks -> _runningTasks for all TrangaTasks 2023-06-27 22:58:40 +02:00
f66ab7d40b Connector use TrangaSettings instead of own values for imageCache and downloadLocation 2023-06-27 22:57:44 +02:00
4cb3694cd5 Re-add task timeout 2023-06-27 22:23:53 +02:00
a05d4c8bd9 Merge remote-tracking branch 'origin/master' 2023-06-27 22:23:23 +02:00
22f87a74b2 Re-add task timeout 2023-06-27 22:23:19 +02:00
ba57282879 Re-add task timeout 2023-06-27 22:19:06 +02:00
9ccba6fba6 Fix CheckChapterIsDownloaded Directory does not exists exception returning 0 chapters 2023-06-25 23:56:22 +02:00
4f01c1166f Fix taskIds being changed during requests, no workaround this time 2023-06-25 23:56:00 +02:00
0a51e7ad3d Fix taskIds being changed during requests 2023-06-25 23:26:36 +02:00
e541b922dc Merge pull request #24 from arxae/master
Added MangaKatana connector
2023-06-25 21:38:18 +02:00
604abd5f9a Fix bug where ChildTasks hung parentTasks 2023-06-24 21:00:26 +02:00
7b311eae75 Will break: CheckChapterIsDownloaded 2023-06-24 20:46:35 +02:00
d4eb72cd99 Required changes 2023-06-23 22:14:27 +02:00
b515215f4b Fix taskIds being changed during requests 2023-06-22 23:09:59 +02:00
a16686dfbf Fix wrong taskNames 2023-06-22 22:52:26 +02:00
4275703941 Added MangaKatana connector 2023-06-22 14:22:21 +02:00
c3342984ea Server fixed bug where ?& in request url caused variables to not parse 2023-06-21 18:04:41 +02:00
ed4bdb5b33 TrangaSettings export after change 2023-06-21 18:04:12 +02:00
0f0902c932 LunaSea changed to id device/id or user/id instead of full url 2023-06-21 18:03:48 +02:00
6508055b43 API Fix closed response socket 2023-06-21 17:42:56 +02:00
abc66511d8 Fixed progress tracking this time for realsies. resolves #5 2023-06-21 17:30:31 +02:00
9ed36c47b5 Fixed taskId on init deserialization 2023-06-21 17:29:48 +02:00
fd1b2a8470 API Fix closed response socket 2023-06-21 17:29:20 +02:00
8058749ab5 Website fix wrong task on deletion 2023-06-21 16:53:56 +02:00
8737617e5f Fix deletion of successful child tasks 2023-06-21 16:53:41 +02:00
7e4f43f1e2 API fix CORS preflight 2023-06-21 16:53:07 +02:00
12b1b2afd6 Server fix interfaces on windows 2023-06-21 16:52:57 +02:00
0f9ac60fcd closes #11 readme update 2023-06-21 16:17:40 +02:00
8c87f2948c README updated screenshots 2023-06-21 16:08:36 +02:00
e0fb817256 Changed glax/tranga-base to latest 2023-06-20 23:26:49 +02:00
43 changed files with 1110 additions and 573 deletions

7
.github/dependabot.yml vendored Normal file
View File

@ -0,0 +1,7 @@
version: 2
updates:
# Maintain dependencies for GitHub Actions
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "daily"

43
.github/workflows/docker-base.yml vendored Normal file
View File

@ -0,0 +1,43 @@
name: Docker Image CI
on:
workflow_dispatch:
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
# https://github.com/docker/setup-qemu-action#usage
- name: Set up QEMU
uses: docker/setup-qemu-action@v2.2.0
# https://github.com/marketplace/actions/docker-setup-buildx
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v2.9.1
# https://github.com/docker/login-action#docker-hub
- name: Login to Docker Hub
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
# https://github.com/docker/build-push-action#multi-platform-image
- name: Build and push base
uses: docker/build-push-action@v4.1.1
with:
context: ./
file: ./Dockerfile-base
#platforms: linux/amd64,linux/arm64,linux/riscv64,linux/ppc64le,linux/s390x,linux/386,linux/mips64le,linux/mips64,linux/arm/v7,linux/arm/v6
platforms: linux/amd64
pull: true
push: true
tags: |
glax/tranga-base:latest

View File

@ -0,0 +1,58 @@
name: Docker Image CI
on:
push:
branches: [ "cuttingedge" ]
workflow_dispatch:
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
# https://github.com/docker/setup-qemu-action#usage
- name: Set up QEMU
uses: docker/setup-qemu-action@v2.2.0
# https://github.com/marketplace/actions/docker-setup-buildx
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v2.9.1
# https://github.com/docker/login-action#docker-hub
- name: Login to Docker Hub
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
# https://github.com/docker/build-push-action#multi-platform-image
- name: Build and push API
uses: docker/build-push-action@v4.1.1
with:
context: ./
file: ./API/Dockerfile
#platforms: linux/amd64,linux/arm64,linux/riscv64,linux/ppc64le,linux/s390x,linux/386,linux/mips64le,linux/mips64,linux/arm/v7,linux/arm/v6
platforms: linux/amd64
pull: true
push: true
tags: |
glax/tranga-api:cuttingedge
# https://github.com/docker/build-push-action#multi-platform-image
- name: Build and push Website
uses: docker/build-push-action@v4.1.1
with:
context: ./Website
file: ./Website/Dockerfile
#platforms: linux/amd64,linux/arm64,linux/riscv64,linux/ppc64le,linux/s390x,linux/386,linux/mips64le,linux/mips64,linux/arm/v7,linux/arm/v6
platforms: linux/amd64
pull: true
push: true
tags: |
glax/tranga-website:cuttingedge

View File

@ -0,0 +1,60 @@
name: Docker Image CI
on:
push:
branches: [ "master" ]
pull_request:
branches: [ "master" ]
workflow_dispatch:
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
# https://github.com/docker/setup-qemu-action#usage
- name: Set up QEMU
uses: docker/setup-qemu-action@v2.2.0
# https://github.com/marketplace/actions/docker-setup-buildx
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v2.9.1
# https://github.com/docker/login-action#docker-hub
- name: Login to Docker Hub
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
# https://github.com/docker/build-push-action#multi-platform-image
- name: Build and push API
uses: docker/build-push-action@v4.1.1
with:
context: ./
file: ./API/Dockerfile
#platforms: linux/amd64,linux/arm64,linux/riscv64,linux/ppc64le,linux/s390x,linux/386,linux/mips64le,linux/mips64,linux/arm/v7,linux/arm/v6
platforms: linux/amd64
pull: true
push: true
tags: |
glax/tranga-api:latest
# https://github.com/docker/build-push-action#multi-platform-image
- name: Build and push Website
uses: docker/build-push-action@v4.1.1
with:
context: ./Website
file: ./Website/Dockerfile
#platforms: linux/amd64,linux/arm64,linux/riscv64,linux/ppc64le,linux/s390x,linux/386,linux/mips64le,linux/mips64,linux/arm/v7,linux/arm/v6
platforms: linux/amd64
pull: true
push: true
tags: |
glax/tranga-website:latest

View File

@ -3,10 +3,10 @@
FROM mcr.microsoft.com/dotnet/sdk:7.0 as build-env FROM mcr.microsoft.com/dotnet/sdk:7.0 as build-env
WORKDIR /src WORKDIR /src
COPY . /src/ COPY . /src/
RUN dotnet restore API/API.csproj RUN dotnet restore /src/API/API.csproj
RUN dotnet publish -c Release -o /publish RUN dotnet publish -c Release -o /publish
FROM glax/tranga-base:dev as runtime FROM glax/tranga-base:latest as runtime
WORKDIR /publish WORKDIR /publish
COPY --from=build-env /publish . COPY --from=build-env /publish .
EXPOSE 6531 EXPOSE 6531

View File

@ -1,10 +1,12 @@
using System.Runtime.InteropServices; using System.Runtime.InteropServices;
using Logging; using Logging;
using Tranga; using Tranga;
using Tranga.NotificationManagers;
using Tranga.LibraryManagers;
namespace API; namespace API;
public class Program public static class Program
{ {
public static void Main(string[] args) public static void Main(string[] args)
{ {
@ -17,28 +19,32 @@ public class Program
Directory.CreateDirectory(logsFolderPath); Directory.CreateDirectory(logsFolderPath);
Logger logger = new(new[] { Logger.LoggerType.FileLogger, Logger.LoggerType.ConsoleLogger }, Console.Out, Console.Out.Encoding, logFilePath); Logger logger = new(new[] { Logger.LoggerType.FileLogger, Logger.LoggerType.ConsoleLogger }, Console.Out, Console.Out.Encoding, logFilePath);
logger.WriteLine("Tranga",value: "\n"+
"-------------------------------------------\n"+
" Starting Tranga-API\n"+
"-------------------------------------------");
logger.WriteLine("Tranga", "Loading settings."); logger.WriteLine("Tranga", "Loading settings.");
TrangaSettings settings; TrangaSettings settings;
if (File.Exists(settingsFilePath)) if (File.Exists(settingsFilePath))
settings = TrangaSettings.LoadSettings(settingsFilePath, logger); settings = TrangaSettings.LoadSettings(settingsFilePath, logger);
else else
settings = new TrangaSettings(downloadFolderPath, applicationFolderPath, new HashSet<LibraryManager>(), new HashSet<NotificationManager>()); settings = new TrangaSettings(downloadFolderPath, applicationFolderPath, new HashSet<LibraryManager>(), new HashSet<NotificationManager>(), logger);
Directory.CreateDirectory(settings.workingDirectory); Directory.CreateDirectory(settings.workingDirectory);
Directory.CreateDirectory(settings.downloadLocation); Directory.CreateDirectory(settings.downloadLocation);
Directory.CreateDirectory(settings.coverImageCache); Directory.CreateDirectory(settings.coverImageCache);
logger.WriteLine("Tranga",$"Application-Folder: {settings.workingDirectory}"); settings.logger?.WriteLine("Tranga",$"Application-Folder: {settings.workingDirectory}");
logger.WriteLine("Tranga",$"Settings-File-Path: {settings.settingsFilePath}"); settings.logger?.WriteLine("Tranga",$"Settings-File-Path: {settings.settingsFilePath}");
logger.WriteLine("Tranga",$"Download-Folder-Path: {settings.downloadLocation}"); settings.logger?.WriteLine("Tranga",$"Download-Folder-Path: {settings.downloadLocation}");
logger.WriteLine("Tranga",$"Logfile-Path: {logFilePath}"); settings.logger?.WriteLine("Tranga",$"Logfile-Path: {logFilePath}");
logger.WriteLine("Tranga",$"Image-Cache-Path: {settings.coverImageCache}"); settings.logger?.WriteLine("Tranga",$"Image-Cache-Path: {settings.coverImageCache}");
logger.WriteLine("Tranga", "Loading Taskmanager."); settings.logger?.WriteLine("Tranga", "Loading Taskmanager.");
TaskManager taskManager = new (settings, logger); TaskManager taskManager = new (settings);
Server server = new (6531, taskManager, logger); Server server = new (6531, taskManager);
foreach(NotificationManager nm in taskManager.settings.notificationManagers) foreach(NotificationManager nm in taskManager.settings.notificationManagers)
nm.SendNotification("Tranga-API", "Started Tranga-API"); nm.SendNotification("Tranga-API", "Started Tranga-API");
} }

View File

@ -1,6 +1,8 @@
using System.Net; using System.Globalization;
using System.Net;
using System.Text.RegularExpressions; using System.Text.RegularExpressions;
using Tranga; using Tranga;
using Tranga.Connectors;
using Tranga.TrangaTasks; using Tranga.TrangaTasks;
namespace API; namespace API;
@ -20,8 +22,8 @@ public class RequestHandler
new[] { "connectorName", "internalId", "onlyNew?", "onlyExisting?", "language?" }), new[] { "connectorName", "internalId", "onlyNew?", "onlyExisting?", "language?" }),
new(HttpMethod.Get, "/Tasks/Types", Array.Empty<string>()), new(HttpMethod.Get, "/Tasks/Types", Array.Empty<string>()),
new(HttpMethod.Post, "/Tasks/CreateMonitorTask", new(HttpMethod.Post, "/Tasks/CreateMonitorTask",
new[] { "connectorName", "internalId", "reoccurrenceTime", "language?" }), new[] { "connectorName", "internalId", "reoccurrenceTime", "language?", "ignoreChaptersBelow?" }),
new(HttpMethod.Post, "/Tasks/CreateUpdateLibraryTask", new[] { "reoccurrenceTime" }), //DEPRECATED new(HttpMethod.Post, "/Tasks/CreateUpdateLibraryTask", new[] { "reoccurrenceTime" }),
new(HttpMethod.Post, "/Tasks/CreateDownloadChaptersTask", new(HttpMethod.Post, "/Tasks/CreateDownloadChaptersTask",
new[] { "connectorName", "internalId", "chapters", "language?" }), new[] { "connectorName", "internalId", "chapters", "language?" }),
new(HttpMethod.Get, "/Tasks", new[] { "taskType", "connectorName?", "publicationId?" }), new(HttpMethod.Get, "/Tasks", new[] { "taskType", "connectorName?", "publicationId?" }),
@ -80,12 +82,17 @@ public class RequestHandler
private Dictionary<string, string> GetRequestVariables(string query) private Dictionary<string, string> GetRequestVariables(string query)
{ {
Dictionary<string, string> ret = new(); Dictionary<string, string> ret = new();
Regex queryRex = new (@"\?{1}([A-z]+=[A-z]+)+(&[A-z]+=[A-z]+)*"); Regex queryRex = new (@"\?{1}&?([A-z0-9-=]+=[A-z0-9-=]+)+(&[A-z0-9-=]+=[A-z0-9-=]+)*");
if (!queryRex.IsMatch(query)) if (!queryRex.IsMatch(query))
return ret; return ret;
query = query.Substring(1); query = query.Substring(1);
foreach(string kvpair in query.Split('&')) foreach (string kvpair in query.Split('&').Where(str => str.Length >= 3))
ret.Add(kvpair.Split('=')[0], kvpair.Split('=')[1]); {
string var = kvpair.Split('=')[0];
string val = Regex.Replace(kvpair.Substring(var.Length + 1), "%20", " ");
val = Regex.Replace(val, "%[0-9]{2}", "");
ret.Add(var, val);
}
return ret; return ret;
} }
@ -146,6 +153,7 @@ public class RequestHandler
variables.TryGetValue("internalId", out string? internalId1); variables.TryGetValue("internalId", out string? internalId1);
variables.TryGetValue("reoccurrenceTime", out string? reoccurrenceTime1); variables.TryGetValue("reoccurrenceTime", out string? reoccurrenceTime1);
variables.TryGetValue("language", out string? language1); variables.TryGetValue("language", out string? language1);
variables.TryGetValue("ignoreChaptersBelow", out string? minChapter);
if (connectorName1 is null || internalId1 is null || reoccurrenceTime1 is null) if (connectorName1 is null || internalId1 is null || reoccurrenceTime1 is null)
return; return;
Connector? connector1 = Connector? connector1 =
@ -153,15 +161,18 @@ public class RequestHandler
if (connector1 is null) if (connector1 is null)
return; return;
Publication? publication1 = _taskManager.GetAllPublications().FirstOrDefault(pub => pub.internalId == internalId1); Publication? publication1 = _taskManager.GetAllPublications().FirstOrDefault(pub => pub.internalId == internalId1);
if (publication1 is null) if (!publication1.HasValue)
return; return;
_taskManager.AddTask(new MonitorPublicationTask(connectorName1, (Publication)publication1, TimeSpan.Parse(reoccurrenceTime1), language1 ?? "en")); Publication pPublication1 = (Publication)publication1;
if (minChapter is not null)
pPublication1.ignoreChaptersBelow = float.Parse(minChapter,new NumberFormatInfo() { NumberDecimalSeparator = "." });
_taskManager.AddTask(new MonitorPublicationTask(connectorName1, pPublication1, TimeSpan.Parse(reoccurrenceTime1), language1 ?? "en"));
break; break;
case "/Tasks/CreateUpdateLibraryTask": case "/Tasks/CreateUpdateLibraryTask": // DEPRECATED
variables.TryGetValue("reoccurrenceTime", out string? reoccurrenceTime2); /*variables.TryGetValue("reoccurrenceTime", out string? reoccurrenceTime2);
if (reoccurrenceTime2 is null) if (reoccurrenceTime2 is null)
return; return;
_taskManager.AddTask(new UpdateLibrariesTask(TimeSpan.Parse(reoccurrenceTime2))); _taskManager.AddTask(new UpdateLibrariesTask(TimeSpan.Parse(reoccurrenceTime2)));*/
break; break;
case "/Tasks/CreateDownloadChaptersTask": case "/Tasks/CreateDownloadChaptersTask":
variables.TryGetValue("connectorName", out string? connectorName2); variables.TryGetValue("connectorName", out string? connectorName2);
@ -178,7 +189,7 @@ public class RequestHandler
if (publication2 is null) if (publication2 is null)
return; return;
IEnumerable<Chapter> toDownload = connector2.SearchChapters((Publication)publication2, chapters, language2 ?? "en"); IEnumerable<Chapter> toDownload = connector2.SelectChapters((Publication)publication2, chapters, language2 ?? "en");
foreach(Chapter chapter in toDownload) foreach(Chapter chapter in toDownload)
_taskManager.AddTask(new DownloadChapterTask(connectorName2, (Publication)publication2, chapter, "en")); _taskManager.AddTask(new DownloadChapterTask(connectorName2, (Publication)publication2, chapter, "en"));
break; break;
@ -236,17 +247,17 @@ public class RequestHandler
variables.TryGetValue("lunaseaWebhook", out string? lunaseaWebhook); variables.TryGetValue("lunaseaWebhook", out string? lunaseaWebhook);
if (downloadLocation is not null && downloadLocation.Length > 0) if (downloadLocation is not null && downloadLocation.Length > 0)
_taskManager.settings.UpdateSettings(TrangaSettings.UpdateField.DownloadLocation, _parent.logger, downloadLocation); _taskManager.settings.UpdateSettings(TrangaSettings.UpdateField.DownloadLocation, downloadLocation);
if (komgaUrl is not null && komgaAuth is not null && komgaUrl.Length > 5 && komgaAuth.Length > 0) if (komgaUrl is not null && komgaAuth is not null && komgaUrl.Length > 5 && komgaAuth.Length > 0)
_taskManager.settings.UpdateSettings(TrangaSettings.UpdateField.Komga, _parent.logger, komgaUrl, komgaAuth); _taskManager.settings.UpdateSettings(TrangaSettings.UpdateField.Komga, komgaUrl, komgaAuth);
if (kavitaUrl is not null && kavitaPassword is not null && kavitaUsername is not null && kavitaUrl.Length > 5 && if (kavitaUrl is not null && kavitaPassword is not null && kavitaUsername is not null && kavitaUrl.Length > 5 &&
kavitaUsername.Length > 0 && kavitaPassword.Length > 0) kavitaUsername.Length > 0 && kavitaPassword.Length > 0)
_taskManager.settings.UpdateSettings(TrangaSettings.UpdateField.Kavita, _parent.logger, kavitaUrl, kavitaUsername, _taskManager.settings.UpdateSettings(TrangaSettings.UpdateField.Kavita, kavitaUrl, kavitaUsername,
kavitaPassword); kavitaPassword);
if (gotifyUrl is not null && gotifyAppToken is not null && gotifyUrl.Length > 5 && gotifyAppToken.Length > 0) if (gotifyUrl is not null && gotifyAppToken is not null && gotifyUrl.Length > 5 && gotifyAppToken.Length > 0)
_taskManager.settings.UpdateSettings(TrangaSettings.UpdateField.Gotify, _parent.logger, gotifyUrl, gotifyAppToken); _taskManager.settings.UpdateSettings(TrangaSettings.UpdateField.Gotify, gotifyUrl, gotifyAppToken);
if(lunaseaWebhook is not null && lunaseaWebhook.Length > 5) if(lunaseaWebhook is not null && lunaseaWebhook.Length > 5)
_taskManager.settings.UpdateSettings(TrangaSettings.UpdateField.LunaSea, _parent.logger, lunaseaWebhook); _taskManager.settings.UpdateSettings(TrangaSettings.UpdateField.LunaSea, lunaseaWebhook);
break; break;
} }
} }
@ -272,7 +283,7 @@ public class RequestHandler
return null; return null;
if(title.Length < 4) if(title.Length < 4)
return null; return null;
return _taskManager.GetPublicationsFromConnector(connector1, title); return connector1.GetPublications(ref _taskManager.collection, title);
case "/Publications/Chapters": case "/Publications/Chapters":
string[] yes = { "true", "yes", "1", "y" }; string[] yes = { "true", "yes", "1", "y" };
variables.TryGetValue("connectorName", out string? connectorName2); variables.TryGetValue("connectorName", out string? connectorName2);
@ -293,7 +304,7 @@ public class RequestHandler
return null; return null;
if(newOnly) if(newOnly)
return _taskManager.GetNewChaptersList(connector2, (Publication)publication, language??"en").ToArray(); return connector2.GetNewChaptersList((Publication)publication, language??"en", ref _taskManager.collection).ToArray();
else if (existingOnly) else if (existingOnly)
return _taskManager.GetExistingChaptersList(connector2, (Publication)publication, language ?? "en").ToArray(); return _taskManager.GetExistingChaptersList(connector2, (Publication)publication, language ?? "en").ToArray();
else else
@ -319,7 +330,7 @@ public class RequestHandler
variables.TryGetValue("taskType", out string? taskType2); variables.TryGetValue("taskType", out string? taskType2);
variables.TryGetValue("connectorName", out string? connectorName4); variables.TryGetValue("connectorName", out string? connectorName4);
variables.TryGetValue("publicationId", out string? publicationId); variables.TryGetValue("publicationId", out string? publicationId);
variables.TryGetValue("chapterSortNumber", out string? chapterSortNumber); variables.TryGetValue("chapterNumber", out string? chapterNumber);
if (taskType2 is null || connectorName4 is null || publicationId is null) if (taskType2 is null || connectorName4 is null || publicationId is null)
return null; return null;
Connector? connector = Connector? connector =
@ -333,10 +344,10 @@ public class RequestHandler
if (pTask is TrangaTask.Task.MonitorPublication) if (pTask is TrangaTask.Task.MonitorPublication)
{ {
task = _taskManager.GetTasksMatching(pTask, connectorName: connectorName4, internalId: publicationId).FirstOrDefault(); task = _taskManager.GetTasksMatching(pTask, connectorName: connectorName4, internalId: publicationId).FirstOrDefault();
}else if (pTask is TrangaTask.Task.DownloadChapter && chapterSortNumber is not null) }else if (pTask is TrangaTask.Task.DownloadChapter && chapterNumber is not null)
{ {
task = _taskManager.GetTasksMatching(pTask, connectorName: connectorName4, internalId: publicationId, task = _taskManager.GetTasksMatching(pTask, connectorName: connectorName4, internalId: publicationId,
chapterSortNumber: chapterSortNumber).FirstOrDefault(); chapterNumber: chapterNumber).FirstOrDefault();
} }
if (task is null) if (task is null)
return null; return null;

View File

@ -1,4 +1,5 @@
using System.Net; using System.Net;
using System.Runtime.InteropServices;
using System.Text; using System.Text;
using System.Text.RegularExpressions; using System.Text.RegularExpressions;
using Logging; using Logging;
@ -18,7 +19,10 @@ public class Server
public Server(int port, TaskManager taskManager, Logger? logger = null) public Server(int port, TaskManager taskManager, Logger? logger = null)
{ {
this.logger = logger; this.logger = logger;
this._listener.Prefixes.Add($"http://*:{port}/"); if(RuntimeInformation.IsOSPlatform(OSPlatform.Linux))
this._listener.Prefixes.Add($"http://*:{port}/");
else
this._listener.Prefixes.Add($"http://localhost:{port}/");
this._requestHandler = new RequestHandler(taskManager, this); this._requestHandler = new RequestHandler(taskManager, this);
Listen(); Listen();
} }
@ -50,8 +54,15 @@ public class Server
SendResponse(HttpStatusCode.BadRequest, response); SendResponse(HttpStatusCode.BadRequest, response);
return; return;
} }
_requestHandler.HandleRequest(request, response); if (request.HttpMethod == "OPTIONS")
{
SendResponse(HttpStatusCode.OK, response);
}
else
{
_requestHandler.HandleRequest(request, response);
}
} }
internal void SendResponse(HttpStatusCode statusCode, HttpListenerResponse response, object? content = null) internal void SendResponse(HttpStatusCode statusCode, HttpListenerResponse response, object? content = null)
@ -63,9 +74,16 @@ public class Server
response.AddHeader("Access-Control-Max-Age", "1728000"); response.AddHeader("Access-Control-Max-Age", "1728000");
response.AppendHeader("Access-Control-Allow-Origin", "*"); response.AppendHeader("Access-Control-Allow-Origin", "*");
response.ContentType = "application/json"; response.ContentType = "application/json";
response.OutputStream.Write(content is not null try
? Encoding.UTF8.GetBytes(JsonConvert.SerializeObject(content)) {
: Array.Empty<byte>()); response.OutputStream.Write(content is not null
? Encoding.UTF8.GetBytes(JsonConvert.SerializeObject(content))
: Array.Empty<byte>());
}
catch (HttpListenerException)
{
}
response.OutputStream.Close(); response.OutputStream.Close();
} }

View File

@ -1,14 +0,0 @@
# syntax=docker/dockerfile:1
FROM mcr.microsoft.com/dotnet/sdk:7.0 as build-env
WORKDIR /src
COPY . /src/
RUN dotnet restore Tranga-API/Tranga-API.csproj
RUN dotnet publish -c Release -o /publish
#FROM mcr.microsoft.com/dotnet/aspnet:7.0 as runtime
FROM glax/tranga-base:latest as runtime
WORKDIR /publish
COPY --from=build-env /publish .
EXPOSE 80
ENTRYPOINT ["dotnet", "/publish/Tranga-API.dll"]

View File

@ -1,5 +1,4 @@
using System.Text; using System.Text;
using System.Text.Json.Serialization;
namespace Logging; namespace Logging;
@ -8,7 +7,7 @@ public class FileLogger : LoggerBase
private string logFilePath { get; } private string logFilePath { get; }
private const int MaxNumberOfLogFiles = 5; private const int MaxNumberOfLogFiles = 5;
public FileLogger(string logFilePath, TextWriter? stdOut, Encoding? encoding = null) : base (stdOut, encoding) public FileLogger(string logFilePath, Encoding? encoding = null) : base (encoding)
{ {
this.logFilePath = logFilePath; this.logFilePath = logFilePath;
@ -22,11 +21,11 @@ public class FileLogger : LoggerBase
{ {
try try
{ {
File.AppendAllText(logFilePath, logMessage.ToString()); File.AppendAllText(logFilePath, logMessage.formattedMessage);
} }
catch (Exception e) catch (Exception)
{ {
stdOut?.WriteLine(e); // ignored
} }
} }
} }

View File

@ -4,14 +4,14 @@ namespace Logging;
public class FormattedConsoleLogger : LoggerBase public class FormattedConsoleLogger : LoggerBase
{ {
private readonly TextWriter _stdOut;
public FormattedConsoleLogger(TextWriter? stdOut, Encoding? encoding = null) : base(stdOut, encoding) public FormattedConsoleLogger(TextWriter stdOut, Encoding? encoding = null) : base(encoding)
{ {
this._stdOut = stdOut;
} }
protected override void Write(LogMessage message) protected override void Write(LogMessage message)
{ {
//Nothing to do yet this._stdOut.Write(message.formattedMessage);
} }
} }

23
Logging/LogMessage.cs Normal file
View File

@ -0,0 +1,23 @@
namespace Logging;
public class LogMessage
{
public DateTime logTime { get; }
public string caller { get; }
public string value { get; }
public string formattedMessage => ToString();
public LogMessage(DateTime messageTime, string caller, string value)
{
this.logTime = messageTime;
this.caller = caller;
this.value = value;
}
public override string ToString()
{
string dateTimeString = $"{logTime.ToShortDateString()} {logTime.ToLongTimeString()}.{logTime.Millisecond,-3}";
string name = caller.Split(new char[] { '.', '+' }).Last();
return $"[{dateTimeString}] {name.Substring(0, name.Length >= 13 ? 13 : name.Length),13} | {value}";
}
}

View File

@ -1,5 +1,4 @@
using System.Net.Mime; using System.Text;
using System.Text;
namespace Logging; namespace Logging;
@ -12,24 +11,31 @@ public class Logger : TextWriter
ConsoleLogger ConsoleLogger
} }
private FileLogger? _fileLogger; private readonly FileLogger? _fileLogger;
private FormattedConsoleLogger? _formattedConsoleLogger; private readonly FormattedConsoleLogger? _formattedConsoleLogger;
private MemoryLogger _memoryLogger; private readonly MemoryLogger _memoryLogger;
private TextWriter? stdOut;
public Logger(LoggerType[] enabledLoggers, TextWriter? stdOut, Encoding? encoding, string? logFilePath) public Logger(LoggerType[] enabledLoggers, TextWriter? stdOut, Encoding? encoding, string? logFilePath)
{ {
this.Encoding = encoding ?? Encoding.ASCII; this.Encoding = encoding ?? Encoding.ASCII;
this.stdOut = stdOut ?? null;
if (enabledLoggers.Contains(LoggerType.FileLogger) && logFilePath is not null) if (enabledLoggers.Contains(LoggerType.FileLogger) && logFilePath is not null)
_fileLogger = new FileLogger(logFilePath, null, encoding); _fileLogger = new FileLogger(logFilePath, encoding);
else else
{ {
_fileLogger = null; _fileLogger = null;
throw new ArgumentException($"logFilePath can not be null for LoggerType {LoggerType.FileLogger}"); throw new ArgumentException($"logFilePath can not be null for LoggerType {LoggerType.FileLogger}");
} }
_formattedConsoleLogger = enabledLoggers.Contains(LoggerType.ConsoleLogger) ? new FormattedConsoleLogger(null, encoding) : null;
_memoryLogger = new MemoryLogger(null, encoding); if (enabledLoggers.Contains(LoggerType.ConsoleLogger) && stdOut is not null)
{
_formattedConsoleLogger = new FormattedConsoleLogger(stdOut, encoding);
}
else
{
_formattedConsoleLogger = null;
throw new ArgumentException($"stdOut can not be null for LoggerType {LoggerType.ConsoleLogger}");
}
_memoryLogger = new MemoryLogger(encoding);
} }
public void WriteLine(string caller, string? value) public void WriteLine(string caller, string? value)
@ -46,9 +52,7 @@ public class Logger : TextWriter
_fileLogger?.Write(caller, value); _fileLogger?.Write(caller, value);
_formattedConsoleLogger?.Write(caller, value); _formattedConsoleLogger?.Write(caller, value);
_memoryLogger.Write(caller, value); _memoryLogger.Write(caller, value);
stdOut?.Write(value);
} }
public string[] Tail(uint? lines) public string[] Tail(uint? lines)

View File

@ -5,21 +5,10 @@ namespace Logging;
public abstract class LoggerBase : TextWriter public abstract class LoggerBase : TextWriter
{ {
public override Encoding Encoding { get; } public override Encoding Encoding { get; }
protected TextWriter? stdOut { get; }
public LoggerBase(TextWriter? stdOut, Encoding? encoding = null) public LoggerBase(Encoding? encoding = null)
{ {
this.Encoding = encoding ?? Encoding.ASCII; this.Encoding = encoding ?? Encoding.ASCII;
this.stdOut = stdOut;
}
public void WriteLine(string caller, string? value)
{
value = value is null ? Environment.NewLine : string.Join(value, Environment.NewLine);
LogMessage message = new LogMessage(DateTime.Now, caller, value);
Write(message);
} }
public void Write(string caller, string? value) public void Write(string caller, string? value)
@ -27,32 +16,10 @@ public abstract class LoggerBase : TextWriter
if (value is null) if (value is null)
return; return;
LogMessage message = new LogMessage(DateTime.Now, caller, value); LogMessage message = new (DateTime.Now, caller, value);
stdOut?.Write(message.ToString());
Write(message); Write(message);
} }
protected abstract void Write(LogMessage message); protected abstract void Write(LogMessage message);
public class LogMessage
{
public DateTime logTime { get; }
public string caller { get; }
public string value { get; }
public LogMessage(DateTime now, string caller, string value)
{
this.logTime = now;
this.caller = caller;
this.value = value;
}
public override string ToString()
{
string dateTimeString = $"{logTime.ToShortDateString()} {logTime.ToLongTimeString()}";
return $"[{dateTimeString}] {caller.Split(new char[]{'.','+'}).Last(),15} | {value}";
}
}
} }

View File

@ -7,7 +7,7 @@ public class MemoryLogger : LoggerBase
private readonly SortedList<DateTime, LogMessage> _logMessages = new(); private readonly SortedList<DateTime, LogMessage> _logMessages = new();
private int _lastLogMessageIndex = 0; private int _lastLogMessageIndex = 0;
public MemoryLogger(TextWriter? stdOut, Encoding? encoding = null) : base(stdOut, encoding) public MemoryLogger(Encoding? encoding = null) : base(encoding)
{ {
} }

View File

@ -52,14 +52,15 @@
<!-- ABOUT THE PROJECT --> <!-- ABOUT THE PROJECT -->
## About The Project ## About The Project
Tranga can download Chapters and Metadata from Scanlation sites such as Tranga can download Chapters and Metadata from "Scanlation" sites such as
- [MangaDex.org](https://mangadex.org/) - [MangaDex.org](https://mangadex.org/)
- [Manganato.com](https://manganato.com/) - [Manganato.com](https://manganato.com/)
- [Mangasee](https://mangasee123.com/) - [Mangasee](https://mangasee123.com/)
- [MangaKatana](https://mangakatana.com)
- ❓ Open an [issue](https://github.com/C9Glax/tranga/issues)
and automatically start updates in [Komga](https://komga.org/) and [Kavita](https://www.kavitareader.com/) to import them. and automatically import them with [Komga](https://komga.org/) and [Kavita](https://www.kavitareader.com/). Also Notifications will be sent to your devices using [Gotify](https://gotify.net/) and [LunaSea](https://www.lunasea.app/).
### Inspiration: ### Inspiration:
Because [Kaizoku](https://github.com/oae/kaizoku) was relying on [mangal](https://github.com/metafates/mangal) and mangal Because [Kaizoku](https://github.com/oae/kaizoku) was relying on [mangal](https://github.com/metafates/mangal) and mangal
@ -76,19 +77,18 @@ That is why I wanted to create my own project, in a language I understand, and t
- Newtonsoft.JSON - Newtonsoft.JSON
- [PuppeteerSharp](https://www.puppeteersharp.com/) - [PuppeteerSharp](https://www.puppeteersharp.com/)
- [Html Agility Pack (HAP)](https://html-agility-pack.net/) - [Html Agility Pack (HAP)](https://html-agility-pack.net/)
- Love <3 Blåhaj 🦈 - 💙 Blåhaj 🦈
<p align="right">(<a href="#readme-top">back to top</a>)</p> <p align="right">(<a href="#readme-top">back to top</a>)</p>
## Screenshots ## Screenshots
![image](screenshots/overview.png) | ![image](screenshots/overview.png) | ![image](screenshots/addtask.png) |
|-----------------------------------:|:----------------------------------|
![image](screenshots/addtask.png) | ![image](screenshots/settings.png) | ![image](screenshots/publication-description.png) | ![image](screenshots/progress.png) |
|-----------------------------------:|:-------------------------------------------------:|:-----------------------------------|
| ![image](screenshots/settings.png) | ![image](screenshots/publication-description.png) |
|-----------------------------------:|:-------------------------------------------------:|
<p align="right">(<a href="#readme-top">back to top</a>)</p> <p align="right">(<a href="#readme-top">back to top</a>)</p>
@ -110,39 +110,42 @@ Download [docker-compose.yaml](https://git.bernloehr.eu/glax/Tranga/src/branch/m
Wherever you are mounting `/usr/share/Tranga-API` you also need to mount that same path + `/imageCache` in the webserver container. Wherever you are mounting `/usr/share/Tranga-API` you also need to mount that same path + `/imageCache` in the webserver container.
### Usage ### Docker-Website usage
There is two ways to download Mangas: There is two ways to download Mangas:
- Downloading everything and monitor for new Chapters - Downloading everything and monitor for new Chapters
- Selecting specific Volumes/Chapters - Selecting specific Volumes/Chapters
On the website you add new tasks, by selecting the blue '+' field. Next select the connector/site you want to use, and enter a search term. On the website you add new tasks, by selecting the blue '+' field. Next select the connector/site you want to use, and enter a search term.
After pressing 'Search', the results will be presented below - this might, depending on the result-size, take a while. After clicking 'Search' (or pressing Enter), the results will be presented below - this might, depending on the result-size, take a while because we are already preloading the cover-images.
Next select the publication and a new popup will open with two options: Next select the publication (by selecting the cover) and a new popup will open with two options:
- "Monitor" - Download all chapters and monitor for new ones - "Monitor" - Download all chapters and monitor for new ones
- "Download Chapter" - Download specific chapters only - "Download Chapter" - Download specific chapters only
When selecting `Monitor` you will be presented with a new window and the selection of the interval you want to check for new chapters. When selecting `Monitor` you will be presented with a new window and the selection of the interval you want to check for new chapters (Default: Every 3 hours).
When selecting `Download Chapter` a list will open with all available chapters from which you can then select a range. When selecting `Download Chapter` a list will open with all available chapters - that have not yet been downloaded - from which you can then select a range (see below).
The syntax for selecting chapters is as follows: The syntax for selecting chapters is as follows:
- To download a single Chapter enter either the index number (the number at the very start of the line) or its absolute number like so: `c(h)(apter)[number]`, spaces are allowed. - To download a single Chapter enter either the index number (the number at the very start of the line) or its absolute number like so: `c(h)(apter)[number]`, spaces are allowed.
- To download a range of chapters enter either a range of index numbers (`3-6`) or chapters (`ch 12-23`). - To download a range of chapters enter either a range of index numbers (`3-6`) or chapters (`ch 12-23`).
- For volumes the syntax is as follows: `v(ol)[number](-[number])`, again spaces allowed. - For volumes the syntax is as follows: `v(ol)[number](-[number])`, again spaces allowed.
Examples: `2-12`, `c1`, `ch 2`, `chapter 3`, `v 2`, `vol3-4`, `v2c4` (note: you can only specify a single chapter with this syntax). Examples: `2-12`, `c1`, `ch 2`, `chapter 3`, `v 2`, `vol3-4`, `v2c4` (note: you can only specify a single chapter with this last syntax).
### Prerequisites ### Prerequisites
#### To Build
[.NET-Core 7.0 SDK](https://dotnet.microsoft.com/en-us/download/dotnet/7.0) [.NET-Core 7.0 SDK](https://dotnet.microsoft.com/en-us/download/dotnet/7.0)
#### To Run
[.NET-Core 7.0 Runtime](https://dotnet.microsoft.com/en-us/download/dotnet/7.0) scroll down a bit, should be on the right the second item.
<!-- ROADMAP --> <!-- ROADMAP -->
## Roadmap ## Roadmap
- [ ] Docker ARM support - [ ] Docker ARM support
- [ ] ? - [ ]
See the [open issues](https://git.bernloehr.eu/glax/Tranga/issues) for a full list of proposed features (and known issues). See the [open issues](https://github.com/C9Glax/tranga/issues) for a full list of proposed features (and known issues).
<p align="right">(<a href="#readme-top">back to top</a>)</p> <p align="right">(<a href="#readme-top">back to top</a>)</p>

View File

@ -1,6 +1,7 @@
using System.Globalization; using System.Globalization;
using Logging; using Logging;
using Tranga; using Tranga;
using Tranga.Connectors;
using Tranga.LibraryManagers; using Tranga.LibraryManagers;
using Tranga.NotificationManagers; using Tranga.NotificationManagers;
using Tranga.TrangaTasks; using Tranga.TrangaTasks;
@ -31,16 +32,16 @@ public static class Tranga_Cli
Logger logger = new(new[] { Logger.LoggerType.FileLogger }, null, Console.Out.Encoding, logFilePath); Logger logger = new(new[] { Logger.LoggerType.FileLogger }, null, Console.Out.Encoding, logFilePath);
logger.WriteLine("Tranga_CLI", "Loading Taskmanager."); logger.WriteLine("Tranga_CLI", "Loading Taskmanager.");
TrangaSettings settings = File.Exists(settingsFilePath) ? TrangaSettings.LoadSettings(settingsFilePath, logger) : new TrangaSettings(Directory.GetCurrentDirectory(), applicationFolderPath, new HashSet<LibraryManager>(), new HashSet<NotificationManager>()); TrangaSettings settings = File.Exists(settingsFilePath) ? TrangaSettings.LoadSettings(settingsFilePath, logger) : new TrangaSettings(Directory.GetCurrentDirectory(), applicationFolderPath, new HashSet<LibraryManager>(), new HashSet<NotificationManager>(), logger);
logger.WriteLine("Tranga_CLI", "User Input"); settings.logger?.WriteLine("Tranga_CLI", "User Input");
Console.WriteLine($"Output folder path [{settings.downloadLocation}]:"); Console.WriteLine($"Output folder path [{settings.downloadLocation}]:");
string? tmpPath = Console.ReadLine(); string? tmpPath = Console.ReadLine();
while(tmpPath is null) while(tmpPath is null)
tmpPath = Console.ReadLine(); tmpPath = Console.ReadLine();
if (tmpPath.Length > 0) if (tmpPath.Length > 0)
settings.UpdateSettings(TrangaSettings.UpdateField.DownloadLocation, logger, tmpPath); settings.UpdateSettings(TrangaSettings.UpdateField.DownloadLocation, tmpPath);
Console.WriteLine($"Komga BaseURL [{settings.libraryManagers.FirstOrDefault(lm => lm.GetType() == typeof(Komga))?.baseUrl}]:"); Console.WriteLine($"Komga BaseURL [{settings.libraryManagers.FirstOrDefault(lm => lm.GetType() == typeof(Komga))?.baseUrl}]:");
string? tmpUrlKomga = Console.ReadLine(); string? tmpUrlKomga = Console.ReadLine();
@ -73,7 +74,7 @@ public static class Tranga_Cli
} }
} while (key != ConsoleKey.Enter); } while (key != ConsoleKey.Enter);
settings.UpdateSettings(TrangaSettings.UpdateField.Komga, logger, tmpUrlKomga, tmpKomgaUser, tmpKomgaPass); settings.UpdateSettings(TrangaSettings.UpdateField.Komga, tmpUrlKomga, tmpKomgaUser, tmpKomgaPass);
} }
Console.WriteLine($"Kavita BaseURL [{settings.libraryManagers.FirstOrDefault(lm => lm.GetType() == typeof(Kavita))?.baseUrl}]:"); Console.WriteLine($"Kavita BaseURL [{settings.libraryManagers.FirstOrDefault(lm => lm.GetType() == typeof(Kavita))?.baseUrl}]:");
@ -107,7 +108,7 @@ public static class Tranga_Cli
} }
} while (key != ConsoleKey.Enter); } while (key != ConsoleKey.Enter);
settings.UpdateSettings(TrangaSettings.UpdateField.Kavita, logger, tmpUrlKavita, tmpKavitaUser, tmpKavitaPass); settings.UpdateSettings(TrangaSettings.UpdateField.Kavita, tmpUrlKavita, tmpKavitaUser, tmpKavitaPass);
} }
Console.WriteLine($"Gotify BaseURL [{((Gotify?)settings.notificationManagers.FirstOrDefault(lm => lm.GetType() == typeof(Gotify)))?.endpoint}]:"); Console.WriteLine($"Gotify BaseURL [{((Gotify?)settings.notificationManagers.FirstOrDefault(lm => lm.GetType() == typeof(Gotify)))?.endpoint}]:");
@ -121,7 +122,7 @@ public static class Tranga_Cli
while (tmpGotifyAppToken is null || tmpGotifyAppToken.Length < 1) while (tmpGotifyAppToken is null || tmpGotifyAppToken.Length < 1)
tmpGotifyAppToken = Console.ReadLine(); tmpGotifyAppToken = Console.ReadLine();
settings.UpdateSettings(TrangaSettings.UpdateField.Gotify, logger, tmpGotifyUrl, tmpGotifyAppToken); settings.UpdateSettings(TrangaSettings.UpdateField.Gotify, tmpGotifyUrl, tmpGotifyAppToken);
} }
logger.WriteLine("Tranga_CLI", "Loaded."); logger.WriteLine("Tranga_CLI", "Loaded.");
@ -132,7 +133,7 @@ public static class Tranga_Cli
private static void TaskMode(TrangaSettings settings, Logger logger) private static void TaskMode(TrangaSettings settings, Logger logger)
{ {
TaskManager taskManager = new (settings, logger); TaskManager taskManager = new (settings);
ConsoleKey selection = ConsoleKey.EraseEndOfFile; ConsoleKey selection = ConsoleKey.EraseEndOfFile;
PrintMenu(taskManager, taskManager.settings.downloadLocation); PrintMenu(taskManager, taskManager.settings.downloadLocation);
while (selection != ConsoleKey.Q) while (selection != ConsoleKey.Q)
@ -504,8 +505,30 @@ public static class Tranga_Cli
Chapter[] availableChapters = connector.GetChapters(publication, "en"); Chapter[] availableChapters = connector.GetChapters(publication, "en");
int cIndex = 0; int cIndex = 0;
Console.WriteLine("Chapters:"); Console.WriteLine("Chapters:");
System.Text.StringBuilder sb = new();
foreach(Chapter chapter in availableChapters) foreach(Chapter chapter in availableChapters)
Console.WriteLine($"{cIndex++}: Vol.{chapter.volumeNumber} Ch.{chapter.chapterNumber} - {chapter.name}"); {
sb.Append($"{cIndex++}: ");
if(string.IsNullOrWhiteSpace(chapter.volumeNumber) == false)
{
sb.Append($"Vol.{chapter.volumeNumber} ");
}
if(string.IsNullOrWhiteSpace(chapter.chapterNumber) == false)
{
sb.Append($"Ch.{chapter.chapterNumber} ");
}
if(string.IsNullOrWhiteSpace(chapter.name) == false)
{
sb.Append($" - {chapter.name}");
}
Console.WriteLine(sb.ToString());
sb.Clear();
}
Console.WriteLine("Enter q to abort"); Console.WriteLine("Enter q to abort");
Console.WriteLine($"Select Chapter(s):"); Console.WriteLine($"Select Chapter(s):");
@ -514,7 +537,7 @@ public static class Tranga_Cli
while(selectedChapters is null || selectedChapters.Length < 1) while(selectedChapters is null || selectedChapters.Length < 1)
selectedChapters = Console.ReadLine(); selectedChapters = Console.ReadLine();
return connector.SearchChapters(publication, selectedChapters); return connector.SelectChapters(publication, selectedChapters);
} }
private static Connector? SelectConnector(Connector[] connectors, Logger logger) private static Connector? SelectConnector(Connector[] connectors, Logger logger)
@ -565,7 +588,7 @@ public static class Tranga_Cli
Console.WriteLine("Publication search query (leave empty for all):"); Console.WriteLine("Publication search query (leave empty for all):");
string? query = Console.ReadLine(); string? query = Console.ReadLine();
Publication[] publications = taskManager.GetPublicationsFromConnector(connector, query ?? ""); Publication[] publications = connector.GetPublications(ref taskManager.collection, query ?? "");
if (publications.Length < 1) if (publications.Length < 1)
{ {

View File

@ -1,5 +1,5 @@
using System.Globalization; using System.Text.RegularExpressions;
using System.Text.RegularExpressions; using System.Xml.Linq;
namespace Tranga; namespace Tranga;
@ -7,34 +7,82 @@ namespace Tranga;
/// Has to be Part of a publication /// Has to be Part of a publication
/// Includes the Chapter-Name, -VolumeNumber, -ChapterNumber, the location of the chapter on the internet and the saveName of the local file. /// Includes the Chapter-Name, -VolumeNumber, -ChapterNumber, the location of the chapter on the internet and the saveName of the local file.
/// </summary> /// </summary>
public struct Chapter public readonly struct Chapter
{ {
public Publication parentPublication { get; }
public string? name { get; } public string? name { get; }
public string? volumeNumber { get; } public string? volumeNumber { get; }
public string? chapterNumber { get; } public string chapterNumber { get; }
public string url { get; } public string url { get; }
public string fileName { get; } public string fileName { get; }
public string sortNumber { get; }
private static readonly Regex LegalCharacters = new Regex(@"([A-z]*[0-9]* *\.*-*,*\]*\[*'*\'*\)*\(*~*!*)*"); private static readonly Regex LegalCharacters = new (@"([A-z]*[0-9]* *\.*-*,*\]*\[*'*\'*\)*\(*~*!*)*");
public Chapter(string? name, string? volumeNumber, string? chapterNumber, string url) private static readonly Regex IllegalStrings = new(@"Vol(ume)?.?", RegexOptions.IgnoreCase);
public Chapter(Publication parentPublication, string? name, string? volumeNumber, string chapterNumber, string url)
{ {
this.parentPublication = parentPublication;
this.name = name; this.name = name;
this.volumeNumber = volumeNumber; this.volumeNumber = volumeNumber;
this.chapterNumber = chapterNumber; this.chapterNumber = chapterNumber;
this.url = url; this.url = url;
NumberFormatInfo nfi = new NumberFormatInfo()
{
NumberDecimalSeparator = "."
};
sortNumber = decimal.Round(Convert.ToDecimal(this.volumeNumber ?? "1") * Convert.ToDecimal(this.chapterNumber, nfi), 1)
.ToString(nfi);
string chapterName = string.Concat(LegalCharacters.Matches(name ?? "")); string chapterName = string.Concat(LegalCharacters.Matches(name ?? ""));
string volStr = this.volumeNumber is not null ? $"Vol.{this.volumeNumber} " : ""; string volStr = this.volumeNumber is not null ? $"Vol.{this.volumeNumber} " : "";
string chNumberStr = this.chapterNumber is not null ? $"Ch.{chapterNumber} " : ""; string chNumberStr = $"Ch.{chapterNumber} ";
string chNameStr = chapterName.Length > 0 ? $"- {chapterName}" : ""; string chNameStr = chapterName.Length > 0 ? $"- {chapterName}" : "";
chNameStr = chNameStr.Replace("Volume", "").Replace("volume", ""); chNameStr = IllegalStrings.Replace(chNameStr, "");
this.fileName = $"{volStr}{chNumberStr}{chNameStr}"; this.fileName = $"{volStr}{chNumberStr}{chNameStr}";
} }
/// <summary>
/// Checks if a chapter-archive is already present
/// </summary>
/// <returns>true if chapter is present</returns>
internal bool CheckChapterIsDownloaded(string downloadLocation)
{
string newFilePath = GetArchiveFilePath(downloadLocation);
if (!Directory.Exists(Path.Join(downloadLocation, parentPublication.folderName)))
return false;
FileInfo[] archives = new DirectoryInfo(Path.Join(downloadLocation, parentPublication.folderName)).GetFiles();
Regex chapterInfoRex = new(@"Ch\.[0-9.]+");
Regex chapterRex = new(@"[0-9]+(\.[0-9]+)?");
if (File.Exists(newFilePath))
return true;
string cn = this.chapterNumber;
if (archives.FirstOrDefault(archive => chapterRex.Match(chapterInfoRex.Match(archive.Name).Value).Value == cn) is { } path)
{
File.Move(path.FullName, newFilePath);
return true;
}
return false;
}
/// <summary>
/// Creates full file path of chapter-archive
/// </summary>
/// <returns>Filepath</returns>
internal string GetArchiveFilePath(string downloadLocation)
{
return Path.Join(downloadLocation, parentPublication.folderName, $"{parentPublication.folderName} - {this.fileName}.cbz");
}
/// <summary>
/// Creates a string containing XML of publication and chapter.
/// See ComicInfo.xml
/// </summary>
/// <returns>XML-string</returns>
internal string GetComicInfoXmlString()
{
XElement comicInfo = new XElement("ComicInfo",
new XElement("Tags", string.Join(',', parentPublication.tags)),
new XElement("LanguageISO", parentPublication.originalLanguage),
new XElement("Title", this.name),
new XElement("Writer", string.Join(',', parentPublication.authors)),
new XElement("Volume", this.volumeNumber),
new XElement("Number", this.chapterNumber)
);
return comicInfo.ToString();
}
} }

View File

@ -1,13 +1,12 @@
using System.IO.Compression; using System.Globalization;
using System.IO.Compression;
using System.Net; using System.Net;
using System.Runtime.InteropServices; using System.Runtime.InteropServices;
using System.Text.RegularExpressions; using System.Text.RegularExpressions;
using System.Xml.Linq;
using Logging;
using Tranga.TrangaTasks; using Tranga.TrangaTasks;
using static System.IO.UnixFileMode; using static System.IO.UnixFileMode;
namespace Tranga; namespace Tranga.Connectors;
/// <summary> /// <summary>
/// Base-Class for all Connectors /// Base-Class for all Connectors
@ -15,27 +14,25 @@ namespace Tranga;
/// </summary> /// </summary>
public abstract class Connector public abstract class Connector
{ {
internal string downloadLocation { get; } //Location of local files protected TrangaSettings settings { get; }
protected DownloadClient downloadClient { get; init; } internal DownloadClient downloadClient { get; init; } = null!;
protected readonly Logger? logger; protected Connector(TrangaSettings settings)
private readonly string _imageCachePath;
protected Connector(string downloadLocation, string imageCachePath, Logger? logger)
{ {
this.downloadLocation = downloadLocation; this.settings = settings;
this.logger = logger; if (!Directory.Exists(settings.coverImageCache))
this.downloadClient = new DownloadClient(new Dictionary<byte, int>() Directory.CreateDirectory(settings.coverImageCache);
{
//RequestTypes for RateLimits
}, logger);
this._imageCachePath = imageCachePath;
if (!Directory.Exists(imageCachePath))
Directory.CreateDirectory(this._imageCachePath);
} }
public abstract string name { get; } //Name of the Connector (e.g. Website) public abstract string name { get; } //Name of the Connector (e.g. Website)
public Publication[] GetPublications(ref HashSet<Publication> publicationCollection, string publicationTitle = "")
{
Publication[] ret = GetPublicationsInternal(publicationTitle);
foreach (Publication p in ret)
publicationCollection.Add(p);
return ret;
}
/// <summary> /// <summary>
/// Returns all Publications with the given string. /// Returns all Publications with the given string.
@ -43,7 +40,7 @@ public abstract class Connector
/// </summary> /// </summary>
/// <param name="publicationTitle">Search-Query</param> /// <param name="publicationTitle">Search-Query</param>
/// <returns>Publications matching the query</returns> /// <returns>Publications matching the query</returns>
public abstract Publication[] GetPublications(string publicationTitle = ""); protected abstract Publication[] GetPublicationsInternal(string publicationTitle = "");
/// <summary> /// <summary>
/// Returns all Chapters of the publication in the provided language. /// Returns all Chapters of the publication in the provided language.
@ -54,18 +51,40 @@ public abstract class Connector
/// <returns>Array of Chapters matching Publication and Language</returns> /// <returns>Array of Chapters matching Publication and Language</returns>
public abstract Chapter[] GetChapters(Publication publication, string language = ""); public abstract Chapter[] GetChapters(Publication publication, string language = "");
public Chapter[] SearchChapters(Publication publication, string searchTerm, string? language = null) /// <summary>
/// Updates the available Chapters of a Publication
/// </summary>
/// <param name="publication">Publication to check</param>
/// <param name="language">Language to receive chapters for</param>
/// <param name="collection"></param>
/// <returns>List of Chapters that were previously not in collection</returns>
public List<Chapter> GetNewChaptersList(Publication publication, string language, ref HashSet<Publication> collection)
{
Chapter[] newChapters = this.GetChapters(publication, language);
collection.Add(publication);
NumberFormatInfo decimalPoint = new (){ NumberDecimalSeparator = "." };
settings.logger?.WriteLine(this.GetType().ToString(), "Checking for duplicates");
List<Chapter> newChaptersList = newChapters.Where(nChapter =>
float.Parse(nChapter.chapterNumber, decimalPoint) > publication.ignoreChaptersBelow &&
!nChapter.CheckChapterIsDownloaded(settings.downloadLocation)).ToList();
settings.logger?.WriteLine(this.GetType().ToString(), $"{newChaptersList.Count} new chapters.");
return newChaptersList;
}
public Chapter[] SelectChapters(Publication publication, string searchTerm, string? language = null)
{ {
Chapter[] availableChapters = this.GetChapters(publication, language??"en"); Chapter[] availableChapters = this.GetChapters(publication, language??"en");
Regex volumeRegex = new ("((v(ol)*(olume)*)+ *([0-9]+(-[0-9]+)?){1})", RegexOptions.IgnoreCase); Regex volumeRegex = new ("((v(ol)*(olume)*)+ *([0-9]+(-[0-9]+)?){1})", RegexOptions.IgnoreCase);
Regex chapterRegex = new ("((c(h)*(hapter)*)+ *([0-9]+(-[0-9]+)?){1})", RegexOptions.IgnoreCase); Regex chapterRegex = new ("((c(h)*(hapter)*)+ *([0-9]+(-[0-9]+)?){1})", RegexOptions.IgnoreCase);
Regex singleResultRegex = new("([0-9]+)", RegexOptions.IgnoreCase); Regex singleResultRegex = new("([0-9]+)", RegexOptions.IgnoreCase);
Regex rangeResultRegex = new("([0-9]+(-[0-9]+))", RegexOptions.IgnoreCase); Regex rangeResultRegex = new("([0-9]+(-[0-9]+))", RegexOptions.IgnoreCase);
Regex allRegex = new("a(ll)?", RegexOptions.IgnoreCase);
if (volumeRegex.IsMatch(searchTerm) && chapterRegex.IsMatch(searchTerm)) if (volumeRegex.IsMatch(searchTerm) && chapterRegex.IsMatch(searchTerm))
{ {
string volume = singleResultRegex.Match(volumeRegex.Match(searchTerm).Value).Value; string volume = singleResultRegex.Match(volumeRegex.Match(searchTerm).Value).Value;
string chapter = singleResultRegex.Match(chapterRegex.Match(searchTerm).Value).Value; string chapter = singleResultRegex.Match(chapterRegex.Match(searchTerm).Value).Value;
return availableChapters.Where(aCh => aCh.volumeNumber is not null && aCh.chapterNumber is not null && return availableChapters.Where(aCh => aCh.volumeNumber is not null &&
aCh.volumeNumber.Equals(volume, StringComparison.InvariantCultureIgnoreCase) && aCh.volumeNumber.Equals(volume, StringComparison.InvariantCultureIgnoreCase) &&
aCh.chapterNumber.Equals(chapter, StringComparison.InvariantCultureIgnoreCase)) aCh.chapterNumber.Equals(chapter, StringComparison.InvariantCultureIgnoreCase))
.ToArray(); .ToArray();
@ -99,15 +118,13 @@ public abstract class Connector
string range = rangeResultRegex.Match(chapter).Value; string range = rangeResultRegex.Match(chapter).Value;
int start = Convert.ToInt32(range.Split('-')[0]); int start = Convert.ToInt32(range.Split('-')[0]);
int end = Convert.ToInt32(range.Split('-')[1]); int end = Convert.ToInt32(range.Split('-')[1]);
return availableChapters.Where(aCh => aCh.chapterNumber is not null && return availableChapters.Where(aCh => Convert.ToInt32(aCh.chapterNumber) >= start &&
Convert.ToInt32(aCh.chapterNumber) >= start &&
Convert.ToInt32(aCh.chapterNumber) <= end).ToArray(); Convert.ToInt32(aCh.chapterNumber) <= end).ToArray();
} }
else if (singleResultRegex.IsMatch(chapter)) else if (singleResultRegex.IsMatch(chapter))
{ {
string chapterNumber = singleResultRegex.Match(chapter).Value; string chapterNumber = singleResultRegex.Match(chapter).Value;
return availableChapters.Where(aCh => return availableChapters.Where(aCh =>
aCh.chapterNumber is not null &&
aCh.chapterNumber.Equals(chapterNumber, StringComparison.InvariantCultureIgnoreCase)).ToArray(); aCh.chapterNumber.Equals(chapterNumber, StringComparison.InvariantCultureIgnoreCase)).ToArray();
} }
} }
@ -121,6 +138,8 @@ public abstract class Connector
} }
else if(singleResultRegex.IsMatch(searchTerm)) else if(singleResultRegex.IsMatch(searchTerm))
return new [] { availableChapters[Convert.ToInt32(searchTerm)] }; return new [] { availableChapters[Convert.ToInt32(searchTerm)] };
else if (allRegex.IsMatch(searchTerm))
return availableChapters;
} }
return Array.Empty<Chapter>(); return Array.Empty<Chapter>();
@ -143,69 +162,24 @@ public abstract class Connector
/// <param name="settings">TrangaSettings</param> /// <param name="settings">TrangaSettings</param>
public void CopyCoverFromCacheToDownloadLocation(Publication publication, TrangaSettings settings) public void CopyCoverFromCacheToDownloadLocation(Publication publication, TrangaSettings settings)
{ {
logger?.WriteLine(this.GetType().ToString(), $"Cloning cover {publication.sortName} -> {publication.internalId}"); settings.logger?.WriteLine(this.GetType().ToString(), $"Cloning cover {publication.sortName} -> {publication.internalId}");
//Check if Publication already has a Folder and cover //Check if Publication already has a Folder and cover
string publicationFolder = publication.CreatePublicationFolder(downloadLocation); string publicationFolder = publication.CreatePublicationFolder(settings.downloadLocation);
DirectoryInfo dirInfo = new (publicationFolder); DirectoryInfo dirInfo = new (publicationFolder);
if (dirInfo.EnumerateFiles().Any(info => info.Name.Contains("cover", StringComparison.InvariantCultureIgnoreCase))) if (dirInfo.EnumerateFiles().Any(info => info.Name.Contains("cover", StringComparison.InvariantCultureIgnoreCase)))
{ {
logger?.WriteLine(this.GetType().ToString(), $"Cover exists {publication.sortName}"); settings.logger?.WriteLine(this.GetType().ToString(), $"Cover exists {publication.sortName}");
return; return;
} }
string fileInCache = Path.Join(settings.coverImageCache, publication.coverFileNameInCache); string fileInCache = Path.Join(settings.coverImageCache, publication.coverFileNameInCache);
string newFilePath = Path.Join(publicationFolder, $"cover.{Path.GetFileName(fileInCache).Split('.')[^1]}" ); string newFilePath = Path.Join(publicationFolder, $"cover.{Path.GetFileName(fileInCache).Split('.')[^1]}" );
logger?.WriteLine(this.GetType().ToString(), $"Cloning cover {fileInCache} -> {newFilePath}"); settings.logger?.WriteLine(this.GetType().ToString(), $"Cloning cover {fileInCache} -> {newFilePath}");
File.Copy(fileInCache, newFilePath, true); File.Copy(fileInCache, newFilePath, true);
if(RuntimeInformation.IsOSPlatform(OSPlatform.Linux)) if(RuntimeInformation.IsOSPlatform(OSPlatform.Linux))
File.SetUnixFileMode(newFilePath, GroupRead | GroupWrite | OtherRead | OtherWrite | UserRead | UserWrite); File.SetUnixFileMode(newFilePath, GroupRead | GroupWrite | OtherRead | OtherWrite | UserRead | UserWrite);
} }
/// <summary>
/// Creates a string containing XML of publication and chapter.
/// See ComicInfo.xml
/// </summary>
/// <returns>XML-string</returns>
protected static string GetComicInfoXmlString(Publication publication, Chapter chapter, Logger? logger)
{
logger?.WriteLine("Connector", $"Creating ComicInfo.Xml for {publication.sortName} {publication.internalId} {chapter.volumeNumber}-{chapter.chapterNumber}");
XElement comicInfo = new XElement("ComicInfo",
new XElement("Tags", string.Join(',',publication.tags)),
new XElement("LanguageISO", publication.originalLanguage),
new XElement("Title", chapter.name),
new XElement("Writer", string.Join(',', publication.authors)),
new XElement("Volume", chapter.volumeNumber),
new XElement("Number", chapter.chapterNumber)
);
return comicInfo.ToString();
}
/// <summary>
/// Checks if a chapter-archive is already present
/// </summary>
/// <returns>true if chapter is present</returns>
public bool CheckChapterIsDownloaded(Publication publication, Chapter chapter)
{
Regex legalCharacters = new Regex(@"([A-z]*[0-9]* *\.*-*,*\]*\[*'*\'*\)*\(*~*!*)*");
string oldFilePath = Path.Join(downloadLocation, publication.folderName, $"{string.Concat(legalCharacters.Matches(chapter.name ?? ""))} - V{chapter.volumeNumber}C{chapter.chapterNumber} - {chapter.sortNumber}.cbz");
string oldFilePath2 = Path.Join(downloadLocation, publication.folderName, $"{string.Concat(legalCharacters.Matches(chapter.name ?? ""))} - VC{chapter.chapterNumber} - {chapter.chapterNumber}.cbz");
string newFilePath = GetArchiveFilePath(publication, chapter);
if (File.Exists(oldFilePath))
File.Move(oldFilePath, newFilePath);
else if (File.Exists(oldFilePath2))
File.Move(oldFilePath2, newFilePath);
return File.Exists(newFilePath);
}
/// <summary>
/// Creates full file path of chapter-archive
/// </summary>
/// <returns>Filepath</returns>
protected string GetArchiveFilePath(Publication publication, Chapter chapter)
{
return Path.Join(downloadLocation, publication.folderName, $"{publication.folderName} - {chapter.fileName}.cbz");
}
/// <summary> /// <summary>
/// Downloads Image from URL and saves it to the given path(incl. fileName) /// Downloads Image from URL and saves it to the given path(incl. fileName)
/// </summary> /// </summary>
@ -238,7 +212,7 @@ public abstract class Connector
{ {
if (cancellationToken?.IsCancellationRequested ?? false) if (cancellationToken?.IsCancellationRequested ?? false)
return HttpStatusCode.RequestTimeout; return HttpStatusCode.RequestTimeout;
logger?.WriteLine("Connector", $"Downloading Images for {saveArchiveFilePath}"); settings.logger?.WriteLine("Connector", $"Downloading Images for {saveArchiveFilePath}");
//Check if Publication Directory already exists //Check if Publication Directory already exists
string directoryPath = Path.GetDirectoryName(saveArchiveFilePath)!; string directoryPath = Path.GetDirectoryName(saveArchiveFilePath)!;
if (!Directory.Exists(directoryPath)) if (!Directory.Exists(directoryPath))
@ -256,7 +230,7 @@ public abstract class Connector
{ {
string[] split = imageUrl.Split('.'); string[] split = imageUrl.Split('.');
string extension = split[^1]; string extension = split[^1];
logger?.WriteLine("Connector", $"Downloading Image {chapter + 1:000}/{imageUrls.Length:000} {parentTask.publication.sortName} {parentTask.publication.internalId} Vol.{parentTask.chapter.volumeNumber} Ch.{parentTask.chapter.chapterNumber} {parentTask.progress:P2}"); settings.logger?.WriteLine("Connector", $"Downloading Image {chapter + 1:000}/{imageUrls.Length:000} {parentTask.publication.sortName} {parentTask.publication.internalId} Vol.{parentTask.chapter.volumeNumber} Ch.{parentTask.chapter.chapterNumber} {parentTask.progress:P2}");
HttpStatusCode status = DownloadImage(imageUrl, Path.Join(tempFolder, $"{chapter++}.{extension}"), requestType, referrer); HttpStatusCode status = DownloadImage(imageUrl, Path.Join(tempFolder, $"{chapter++}.{extension}"), requestType, referrer);
if ((int)status < 200 || (int)status >= 300) if ((int)status < 200 || (int)status >= 300)
return status; return status;
@ -268,7 +242,7 @@ public abstract class Connector
if(comicInfoPath is not null) if(comicInfoPath is not null)
File.Copy(comicInfoPath, Path.Join(tempFolder, "ComicInfo.xml")); File.Copy(comicInfoPath, Path.Join(tempFolder, "ComicInfo.xml"));
logger?.WriteLine("Connector", $"Creating archive {saveArchiveFilePath}"); settings.logger?.WriteLine("Connector", $"Creating archive {saveArchiveFilePath}");
//ZIP-it and ship-it //ZIP-it and ship-it
ZipFile.CreateFromDirectory(tempFolder, saveArchiveFilePath); ZipFile.CreateFromDirectory(tempFolder, saveArchiveFilePath);
if(RuntimeInformation.IsOSPlatform(OSPlatform.Linux)) if(RuntimeInformation.IsOSPlatform(OSPlatform.Linux))
@ -281,7 +255,7 @@ public abstract class Connector
{ {
string[] split = url.Split('/'); string[] split = url.Split('/');
string filename = split[^1]; string filename = split[^1];
string saveImagePath = Path.Join(_imageCachePath, filename); string saveImagePath = Path.Join(settings.coverImageCache, filename);
if (File.Exists(saveImagePath)) if (File.Exists(saveImagePath))
return filename; return filename;
@ -290,95 +264,7 @@ public abstract class Connector
using MemoryStream ms = new(); using MemoryStream ms = new();
coverResult.result.CopyTo(ms); coverResult.result.CopyTo(ms);
File.WriteAllBytes(saveImagePath, ms.ToArray()); File.WriteAllBytes(saveImagePath, ms.ToArray());
logger?.WriteLine(this.GetType().ToString(), $"Saving image to {saveImagePath}"); settings.logger?.WriteLine(this.GetType().ToString(), $"Saving image to {saveImagePath}");
return filename; return filename;
} }
protected class DownloadClient
{
private static readonly HttpClient Client = new()
{
Timeout = TimeSpan.FromSeconds(60)
};
private readonly Dictionary<byte, DateTime> _lastExecutedRateLimit;
private readonly Dictionary<byte, TimeSpan> _rateLimit;
// ReSharper disable once InconsistentNaming
private readonly Logger? logger;
/// <summary>
/// Creates a httpClient
/// </summary>
/// <param name="rateLimitRequestsPerMinute">Rate limits for requests. byte is RequestType, int maximum requests per minute for RequestType</param>
/// <param name="logger"></param>
public DownloadClient(Dictionary<byte, int> rateLimitRequestsPerMinute, Logger? logger)
{
this.logger = logger;
_lastExecutedRateLimit = new();
_rateLimit = new();
foreach(KeyValuePair<byte, int> limit in rateLimitRequestsPerMinute)
_rateLimit.Add(limit.Key, TimeSpan.FromMinutes(1).Divide(limit.Value));
}
/// <summary>
/// Request Webpage
/// </summary>
/// <param name="url"></param>
/// <param name="requestType">For RateLimits: Same Endpoints use same type</param>
/// <param name="referrer">Used in http request header</param>
/// <returns>RequestResult with StatusCode and Stream of received data</returns>
public RequestResult MakeRequest(string url, byte requestType, string? referrer = null)
{
if (_rateLimit.TryGetValue(requestType, out TimeSpan value))
_lastExecutedRateLimit.TryAdd(requestType, DateTime.Now.Subtract(value));
else
{
logger?.WriteLine(this.GetType().ToString(), "RequestType not configured for rate-limit.");
return new RequestResult(HttpStatusCode.NotAcceptable, Stream.Null);
}
TimeSpan rateLimitTimeout = _rateLimit[requestType]
.Subtract(DateTime.Now.Subtract(_lastExecutedRateLimit[requestType]));
if(rateLimitTimeout > TimeSpan.Zero)
Thread.Sleep(rateLimitTimeout);
HttpResponseMessage? response = null;
while (response is null)
{
try
{
HttpRequestMessage requestMessage = new(HttpMethod.Get, url);
if(referrer is not null)
requestMessage.Headers.Referrer = new Uri(referrer);
_lastExecutedRateLimit[requestType] = DateTime.Now;
response = Client.Send(requestMessage);
}
catch (HttpRequestException e)
{
logger?.WriteLine(this.GetType().ToString(), e.Message);
logger?.WriteLine(this.GetType().ToString(), $"Waiting {_rateLimit[requestType] * 2}... Retrying.");
Thread.Sleep(_rateLimit[requestType] * 2);
}
}
if (!response.IsSuccessStatusCode)
{
logger?.WriteLine(this.GetType().ToString(), $"Request-Error {response.StatusCode}: {response.ReasonPhrase}");
return new RequestResult(response.StatusCode, Stream.Null);
}
return new RequestResult(response.StatusCode, response.Content.ReadAsStream());
}
public struct RequestResult
{
public HttpStatusCode statusCode { get; }
public Stream result { get; }
public RequestResult(HttpStatusCode statusCode, Stream result)
{
this.statusCode = statusCode;
this.result = result;
}
}
}
} }

View File

@ -2,7 +2,6 @@
using System.Net; using System.Net;
using System.Text.Json; using System.Text.Json;
using System.Text.Json.Nodes; using System.Text.Json.Nodes;
using Logging;
using Tranga.TrangaTasks; using Tranga.TrangaTasks;
namespace Tranga.Connectors; namespace Tranga.Connectors;
@ -19,7 +18,7 @@ public class MangaDex : Connector
Author, Author,
} }
public MangaDex(string downloadLocation, string imageCachePath, Logger? logger) : base(downloadLocation, imageCachePath, logger) public MangaDex(TrangaSettings settings) : base(settings)
{ {
name = "MangaDex"; name = "MangaDex";
this.downloadClient = new DownloadClient(new Dictionary<byte, int>() this.downloadClient = new DownloadClient(new Dictionary<byte, int>()
@ -29,12 +28,12 @@ public class MangaDex : Connector
{(byte)RequestType.AtHomeServer, 40}, {(byte)RequestType.AtHomeServer, 40},
{(byte)RequestType.CoverUrl, 250}, {(byte)RequestType.CoverUrl, 250},
{(byte)RequestType.Author, 250} {(byte)RequestType.Author, 250}
}, logger); }, settings.logger);
} }
public override Publication[] GetPublications(string publicationTitle = "") protected override Publication[] GetPublicationsInternal(string publicationTitle = "")
{ {
logger?.WriteLine(this.GetType().ToString(), $"Getting Publications (title={publicationTitle})"); settings.logger?.WriteLine(this.GetType().ToString(), $"Getting Publications (title={publicationTitle})");
const int limit = 100; //How many values we want returned at once const int limit = 100; //How many values we want returned at once
int offset = 0; //"Page" int offset = 0; //"Page"
int total = int.MaxValue; //How many total results are there, is updated on first request int total = int.MaxValue; //How many total results are there, is updated on first request
@ -60,7 +59,7 @@ public class MangaDex : Connector
//Loop each Manga and extract information from JSON //Loop each Manga and extract information from JSON
foreach (JsonNode? mangeNode in mangaInResult) foreach (JsonNode? mangeNode in mangaInResult)
{ {
logger?.WriteLine(this.GetType().ToString(), $"Getting publication data. {++loadedPublicationData}/{total}"); settings.logger?.WriteLine(this.GetType().ToString(), $"Getting publication data. {++loadedPublicationData}/{total}");
JsonObject manga = (JsonObject)mangeNode!; JsonObject manga = (JsonObject)mangeNode!;
JsonObject attributes = manga["attributes"]!.AsObject(); JsonObject attributes = manga["attributes"]!.AsObject();
@ -147,13 +146,13 @@ public class MangaDex : Connector
} }
} }
logger?.WriteLine(this.GetType().ToString(), $"Done getting publications (title={publicationTitle})"); settings.logger?.WriteLine(this.GetType().ToString(), $"Done getting publications (title={publicationTitle})");
return publications.ToArray(); return publications.ToArray();
} }
public override Chapter[] GetChapters(Publication publication, string language = "") public override Chapter[] GetChapters(Publication publication, string language = "")
{ {
logger?.WriteLine(this.GetType().ToString(), $"Getting Chapters for {publication.sortName} {publication.internalId} (language={language})"); settings.logger?.WriteLine(this.GetType().ToString(), $"Getting Chapters for {publication.sortName} {publication.internalId} (language={language})");
const int limit = 100; //How many values we want returned at once const int limit = 100; //How many values we want returned at once
int offset = 0; //"Page" int offset = 0; //"Page"
int total = int.MaxValue; //How many total results are there, is updated on first request int total = int.MaxValue; //How many total results are there, is updated on first request
@ -190,11 +189,12 @@ public class MangaDex : Connector
? attributes["volume"]!.GetValue<string>() ? attributes["volume"]!.GetValue<string>()
: null; : null;
string? chapterNum = attributes.ContainsKey("chapter") && attributes["chapter"] is not null string chapterNum = attributes.ContainsKey("chapter") && attributes["chapter"] is not null
? attributes["chapter"]!.GetValue<string>() ? attributes["chapter"]!.GetValue<string>()
: null; : "null";
chapters.Add(new Chapter(title, volume, chapterNum, chapterId)); if(chapterNum is not "null")
chapters.Add(new Chapter(publication, title, volume, chapterNum, chapterId));
} }
} }
@ -203,7 +203,7 @@ public class MangaDex : Connector
{ {
NumberDecimalSeparator = "." NumberDecimalSeparator = "."
}; };
logger?.WriteLine(this.GetType().ToString(), $"Done getting Chapters for {publication.internalId}"); settings.logger?.WriteLine(this.GetType().ToString(), $"Done getting {chapters.Count} Chapters for {publication.internalId}");
return chapters.OrderBy(chapter => Convert.ToSingle(chapter.chapterNumber, chapterNumberFormatInfo)).ToArray(); return chapters.OrderBy(chapter => Convert.ToSingle(chapter.chapterNumber, chapterNumberFormatInfo)).ToArray();
} }
@ -211,7 +211,7 @@ public class MangaDex : Connector
{ {
if (cancellationToken?.IsCancellationRequested ?? false) if (cancellationToken?.IsCancellationRequested ?? false)
return HttpStatusCode.RequestTimeout; return HttpStatusCode.RequestTimeout;
logger?.WriteLine(this.GetType().ToString(), $"Downloading Chapter-Info {publication.sortName} {publication.internalId} {chapter.volumeNumber}-{chapter.chapterNumber}"); settings.logger?.WriteLine(this.GetType().ToString(), $"Downloading Chapter-Info {publication.sortName} {publication.internalId} {chapter.volumeNumber}-{chapter.chapterNumber}");
//Request URLs for Chapter-Images //Request URLs for Chapter-Images
DownloadClient.RequestResult requestResult = DownloadClient.RequestResult requestResult =
downloadClient.MakeRequest($"https://api.mangadex.org/at-home/server/{chapter.url}?forcePort443=false'", (byte)RequestType.AtHomeServer); downloadClient.MakeRequest($"https://api.mangadex.org/at-home/server/{chapter.url}?forcePort443=false'", (byte)RequestType.AtHomeServer);
@ -230,18 +230,18 @@ public class MangaDex : Connector
imageUrls.Add($"{baseUrl}/data/{hash}/{image!.GetValue<string>()}"); imageUrls.Add($"{baseUrl}/data/{hash}/{image!.GetValue<string>()}");
string comicInfoPath = Path.GetTempFileName(); string comicInfoPath = Path.GetTempFileName();
File.WriteAllText(comicInfoPath, GetComicInfoXmlString(publication, chapter, logger)); File.WriteAllText(comicInfoPath, chapter.GetComicInfoXmlString());
//Download Chapter-Images //Download Chapter-Images
return DownloadChapterImages(imageUrls.ToArray(), GetArchiveFilePath(publication, chapter), (byte)RequestType.AtHomeServer, parentTask, comicInfoPath, cancellationToken:cancellationToken); return DownloadChapterImages(imageUrls.ToArray(), chapter.GetArchiveFilePath(settings.downloadLocation), (byte)RequestType.AtHomeServer, parentTask, comicInfoPath, cancellationToken:cancellationToken);
} }
private string? GetCoverUrl(string publicationId, string? posterId) private string? GetCoverUrl(string publicationId, string? posterId)
{ {
logger?.WriteLine(this.GetType().ToString(), $"Getting CoverUrl for {publicationId}"); settings.logger?.WriteLine(this.GetType().ToString(), $"Getting CoverUrl for {publicationId}");
if (posterId is null) if (posterId is null)
{ {
logger?.WriteLine(this.GetType().ToString(), $"No posterId, aborting"); settings.logger?.WriteLine(this.GetType().ToString(), $"No posterId, aborting");
return null; return null;
} }
@ -257,7 +257,7 @@ public class MangaDex : Connector
string fileName = result["data"]!["attributes"]!["fileName"]!.GetValue<string>(); string fileName = result["data"]!["attributes"]!["fileName"]!.GetValue<string>();
string coverUrl = $"https://uploads.mangadex.org/covers/{publicationId}/{fileName}"; string coverUrl = $"https://uploads.mangadex.org/covers/{publicationId}/{fileName}";
logger?.WriteLine(this.GetType().ToString(), $"Got Cover-Url for {publicationId} -> {coverUrl}"); settings.logger?.WriteLine(this.GetType().ToString(), $"Got Cover-Url for {publicationId} -> {coverUrl}");
return coverUrl; return coverUrl;
} }
@ -276,7 +276,7 @@ public class MangaDex : Connector
string authorName = result["data"]!["attributes"]!["name"]!.GetValue<string>(); string authorName = result["data"]!["attributes"]!["name"]!.GetValue<string>();
ret.Add(authorName); ret.Add(authorName);
logger?.WriteLine(this.GetType().ToString(), $"Got author {authorId} -> {authorName}"); settings.logger?.WriteLine(this.GetType().ToString(), $"Got author {authorId} -> {authorName}");
} }
return ret; return ret;
} }

View File

@ -0,0 +1,217 @@
using System.Globalization;
using System.Net;
using System.Text.RegularExpressions;
using HtmlAgilityPack;
using Tranga.TrangaTasks;
namespace Tranga.Connectors;
public class MangaKatana : Connector
{
public override string name { get; }
public MangaKatana(TrangaSettings settings) : base(settings)
{
this.name = "MangaKatana";
this.downloadClient = new DownloadClient(new Dictionary<byte, int>()
{
{(byte)1, 60}
}, settings.logger);
}
protected override Publication[] GetPublicationsInternal(string publicationTitle = "")
{
settings.logger?.WriteLine(this.GetType().ToString(), $"Getting Publications (title={publicationTitle})");
string sanitizedTitle = string.Concat(Regex.Matches(publicationTitle, "[A-z]* *")).ToLower().Replace(' ', '_');
string requestUrl = $"https://mangakatana.com/?search={sanitizedTitle}&search_by=book_name";
DownloadClient.RequestResult requestResult =
downloadClient.MakeRequest(requestUrl, (byte)1);
if ((int)requestResult.statusCode < 200 || (int)requestResult.statusCode >= 300)
return Array.Empty<Publication>();
// If a single result is found, the user will be redirected to the results directly instead of a result page
if(requestResult.HasBeenRedirected
&& requestResult.RedirectedToUrl is not null
&& requestResult.RedirectedToUrl.Contains("mangakatana.com/manga"))
{
return new Publication[] { ParseSinglePublicationFromHtml(requestResult.result, requestResult.RedirectedToUrl.Split('/')[^1]) };
}
return ParsePublicationsFromHtml(requestResult.result);
}
private Publication[] ParsePublicationsFromHtml(Stream html)
{
StreamReader reader = new(html);
string htmlString = reader.ReadToEnd();
HtmlDocument document = new();
document.LoadHtml(htmlString);
IEnumerable<HtmlNode> searchResults = document.DocumentNode.SelectNodes("//*[@id='book_list']/div");
List<string> urls = new();
foreach (HtmlNode mangaResult in searchResults)
{
urls.Add(mangaResult.Descendants("a").First().GetAttributes()
.First(a => a.Name == "href").Value);
}
HashSet<Publication> ret = new();
foreach (string url in urls)
{
DownloadClient.RequestResult requestResult =
downloadClient.MakeRequest(url, (byte)1);
if ((int)requestResult.statusCode < 200 || (int)requestResult.statusCode >= 300)
return Array.Empty<Publication>();
ret.Add(ParseSinglePublicationFromHtml(requestResult.result, url.Split('/')[^1]));
}
return ret.ToArray();
}
private Publication ParseSinglePublicationFromHtml(Stream html, string publicationId)
{
StreamReader reader = new(html);
string htmlString = reader.ReadToEnd();
HtmlDocument document = new();
document.LoadHtml(htmlString);
string status = "";
Dictionary<string, string> altTitles = new();
Dictionary<string, string>? links = null;
HashSet<string> tags = new();
string[] authors = Array.Empty<string>();
string originalLanguage = "";
HtmlNode infoNode = document.DocumentNode.SelectSingleNode("//*[@id='single_book']");
string sortName = infoNode.Descendants("h1").First(n => n.HasClass("heading")).InnerText;
HtmlNode infoTable = infoNode.SelectSingleNode("//*[@id='single_book']/div[2]/div/ul");
foreach (HtmlNode row in infoTable.Descendants("li"))
{
string key = row.SelectNodes("div").First().InnerText.ToLower();
string value = row.SelectNodes("div").Last().InnerText;
string keySanitized = string.Concat(Regex.Matches(key, "[a-z]"));
switch (keySanitized)
{
case "altnames":
string[] alts = value.Split(" ; ");
for (int i = 0; i < alts.Length; i++)
altTitles.Add(i.ToString(), alts[i]);
break;
case "authorsartists":
authors = value.Split(',');
break;
case "status":
status = value;
break;
case "genres":
tags = row.SelectNodes("div").Last().Descendants("a").Select(a => a.InnerText).ToHashSet();
break;
default: break;
}
}
string posterUrl = document.DocumentNode.SelectSingleNode("//*[@id='single_book']/div[1]/div").Descendants("img").First()
.GetAttributes().First(a => a.Name == "src").Value;
string coverFileNameInCache = SaveCoverImageToCache(posterUrl, 1);
string description = document.DocumentNode.SelectSingleNode("//*[@id='single_book']/div[3]/p").InnerText;
while (description.StartsWith('\n'))
description = description.Substring(1);
int year = DateTime.Now.Year;
string yearString = infoTable.Descendants("div").First(d => d.HasClass("updateAt"))
.InnerText.Split('-')[^1];
if(yearString.Contains("ago") == false)
{
year = Convert.ToInt32(yearString);
}
return new Publication(sortName, authors.ToList(), description, altTitles, tags.ToArray(), posterUrl, coverFileNameInCache, links,
year, originalLanguage, status, publicationId);
}
public override Chapter[] GetChapters(Publication publication, string language = "")
{
settings.logger?.WriteLine(this.GetType().ToString(), $"Getting Chapters for {publication.sortName} {publication.internalId} (language={language})");
string requestUrl = $"https://mangakatana.com/manga/{publication.publicationId}";
// Leaving this in for verification if the page exists
DownloadClient.RequestResult requestResult =
downloadClient.MakeRequest(requestUrl, (byte)1);
if ((int)requestResult.statusCode < 200 || (int)requestResult.statusCode >= 300)
return Array.Empty<Chapter>();
//Return Chapters ordered by Chapter-Number
NumberFormatInfo chapterNumberFormatInfo = new()
{
NumberDecimalSeparator = "."
};
List<Chapter> chapters = ParseChaptersFromHtml(publication, requestUrl);
settings.logger?.WriteLine(this.GetType().ToString(), $"Done getting Chapters for {publication.internalId}");
return chapters.OrderBy(chapter => Convert.ToSingle(chapter.chapterNumber, chapterNumberFormatInfo)).ToArray();
}
private List<Chapter> ParseChaptersFromHtml(Publication publication, string mangaUrl)
{
// Using HtmlWeb will include the chapters since they are loaded with js
HtmlWeb web = new();
HtmlDocument document = web.Load(mangaUrl);
List<Chapter> ret = new();
HtmlNode chapterList = document.DocumentNode.SelectSingleNode("//div[contains(@class, 'chapters')]/table/tbody");
foreach (HtmlNode chapterInfo in chapterList.Descendants("tr"))
{
string fullString = chapterInfo.Descendants("a").First().InnerText;
string? volumeNumber = fullString.Contains("Vol.") ? fullString.Replace("Vol.", "").Split(' ')[0] : null;
string? chapterNumber = fullString.Split(':')[0].Split("Chapter ")[1].Split(" ")[0].Replace('-', '.');
string chapterName = string.Concat(fullString.Split(':')[1..]);
string url = chapterInfo.Descendants("a").First()
.GetAttributeValue("href", "");
ret.Add(new Chapter(publication, chapterName, volumeNumber, chapterNumber, url));
}
return ret;
}
public override HttpStatusCode DownloadChapter(Publication publication, Chapter chapter, DownloadChapterTask parentTask, CancellationToken? cancellationToken = null)
{
if (cancellationToken?.IsCancellationRequested ?? false)
return HttpStatusCode.RequestTimeout;
settings.logger?.WriteLine(this.GetType().ToString(), $"Downloading Chapter-Info {publication.sortName} {publication.internalId} {chapter.volumeNumber}-{chapter.chapterNumber}");
string requestUrl = chapter.url;
// Leaving this in to check if the page exists
DownloadClient.RequestResult requestResult =
downloadClient.MakeRequest(requestUrl, (byte)1);
if ((int)requestResult.statusCode < 200 || (int)requestResult.statusCode >= 300)
return requestResult.statusCode;
string[] imageUrls = ParseImageUrlsFromHtml(requestUrl);
string comicInfoPath = Path.GetTempFileName();
File.WriteAllText(comicInfoPath, chapter.GetComicInfoXmlString());
return DownloadChapterImages(imageUrls, chapter.GetArchiveFilePath(settings.downloadLocation), (byte)1, parentTask, comicInfoPath, "https://mangakatana.com/", cancellationToken);
}
private string[] ParseImageUrlsFromHtml(string mangaUrl)
{
HtmlWeb web = new();
HtmlDocument document = web.Load(mangaUrl);
// Images are loaded dynamically, but the urls are present in a piece of js code on the page
string js = document.DocumentNode.SelectSingleNode("//script[contains(., 'data-src')]").InnerText
.Replace("\r", "")
.Replace("\n", "")
.Replace("\t", "");
string regexPat = @"(var thzq=\[')(.*)(,];function)";
var group = Regex.Matches(js, regexPat).First().Groups[2].Value.Replace("'", "");
var urls = group.Split(',');
return urls;
}
}

View File

@ -2,7 +2,6 @@
using System.Net; using System.Net;
using System.Text.RegularExpressions; using System.Text.RegularExpressions;
using HtmlAgilityPack; using HtmlAgilityPack;
using Logging;
using Tranga.TrangaTasks; using Tranga.TrangaTasks;
namespace Tranga.Connectors; namespace Tranga.Connectors;
@ -11,19 +10,19 @@ public class Manganato : Connector
{ {
public override string name { get; } public override string name { get; }
public Manganato(string downloadLocation, string imageCachePath, Logger? logger) : base(downloadLocation, imageCachePath, logger) public Manganato(TrangaSettings settings) : base(settings)
{ {
this.name = "Manganato"; this.name = "Manganato";
this.downloadClient = new DownloadClient(new Dictionary<byte, int>() this.downloadClient = new DownloadClient(new Dictionary<byte, int>()
{ {
{(byte)1, 60} {(byte)1, 60}
}, logger); }, settings.logger);
} }
public override Publication[] GetPublications(string publicationTitle = "") protected override Publication[] GetPublicationsInternal(string publicationTitle = "")
{ {
logger?.WriteLine(this.GetType().ToString(), $"Getting Publications (title={publicationTitle})"); settings.logger?.WriteLine(this.GetType().ToString(), $"Getting Publications (title={publicationTitle})");
string sanitizedTitle = string.Concat(Regex.Matches(publicationTitle, "[A-z]* *")).ToLower().Replace(' ', '_'); string sanitizedTitle = string.Join('_', Regex.Matches(publicationTitle, "[A-z]*")).ToLower();
string requestUrl = $"https://manganato.com/search/story/{sanitizedTitle}"; string requestUrl = $"https://manganato.com/search/story/{sanitizedTitle}";
DownloadClient.RequestResult requestResult = DownloadClient.RequestResult requestResult =
downloadClient.MakeRequest(requestUrl, (byte)1); downloadClient.MakeRequest(requestUrl, (byte)1);
@ -127,7 +126,7 @@ public class Manganato : Connector
public override Chapter[] GetChapters(Publication publication, string language = "") public override Chapter[] GetChapters(Publication publication, string language = "")
{ {
logger?.WriteLine(this.GetType().ToString(), $"Getting Chapters for {publication.sortName} {publication.internalId} (language={language})"); settings.logger?.WriteLine(this.GetType().ToString(), $"Getting Chapters for {publication.sortName} {publication.internalId} (language={language})");
string requestUrl = $"https://chapmanganato.com/{publication.publicationId}"; string requestUrl = $"https://chapmanganato.com/{publication.publicationId}";
DownloadClient.RequestResult requestResult = DownloadClient.RequestResult requestResult =
downloadClient.MakeRequest(requestUrl, (byte)1); downloadClient.MakeRequest(requestUrl, (byte)1);
@ -139,12 +138,12 @@ public class Manganato : Connector
{ {
NumberDecimalSeparator = "." NumberDecimalSeparator = "."
}; };
List<Chapter> chapters = ParseChaptersFromHtml(requestResult.result); List<Chapter> chapters = ParseChaptersFromHtml(publication, requestResult.result);
logger?.WriteLine(this.GetType().ToString(), $"Done getting Chapters for {publication.internalId}"); settings.logger?.WriteLine(this.GetType().ToString(), $"Done getting Chapters for {publication.internalId}");
return chapters.OrderBy(chapter => Convert.ToSingle(chapter.chapterNumber, chapterNumberFormatInfo)).ToArray(); return chapters.OrderBy(chapter => Convert.ToSingle(chapter.chapterNumber, chapterNumberFormatInfo)).ToArray();
} }
private List<Chapter> ParseChaptersFromHtml(Stream html) private List<Chapter> ParseChaptersFromHtml(Publication publication, Stream html)
{ {
StreamReader reader = new (html); StreamReader reader = new (html);
string htmlString = reader.ReadToEnd(); string htmlString = reader.ReadToEnd();
@ -163,7 +162,7 @@ public class Manganato : Connector
string chapterName = string.Concat(fullString.Split(':')[1..]); string chapterName = string.Concat(fullString.Split(':')[1..]);
string url = chapterInfo.Descendants("a").First(d => d.HasClass("chapter-name")) string url = chapterInfo.Descendants("a").First(d => d.HasClass("chapter-name"))
.GetAttributeValue("href", ""); .GetAttributeValue("href", "");
ret.Add(new Chapter(chapterName, volumeNumber, chapterNumber, url)); ret.Add(new Chapter(publication, chapterName, volumeNumber, chapterNumber, url));
} }
ret.Reverse(); ret.Reverse();
return ret; return ret;
@ -173,7 +172,7 @@ public class Manganato : Connector
{ {
if (cancellationToken?.IsCancellationRequested ?? false) if (cancellationToken?.IsCancellationRequested ?? false)
return HttpStatusCode.RequestTimeout; return HttpStatusCode.RequestTimeout;
logger?.WriteLine(this.GetType().ToString(), $"Downloading Chapter-Info {publication.sortName} {publication.internalId} {chapter.volumeNumber}-{chapter.chapterNumber}"); settings.logger?.WriteLine(this.GetType().ToString(), $"Downloading Chapter-Info {publication.sortName} {publication.internalId} {chapter.volumeNumber}-{chapter.chapterNumber}");
string requestUrl = chapter.url; string requestUrl = chapter.url;
DownloadClient.RequestResult requestResult = DownloadClient.RequestResult requestResult =
downloadClient.MakeRequest(requestUrl, (byte)1); downloadClient.MakeRequest(requestUrl, (byte)1);
@ -183,9 +182,9 @@ public class Manganato : Connector
string[] imageUrls = ParseImageUrlsFromHtml(requestResult.result); string[] imageUrls = ParseImageUrlsFromHtml(requestResult.result);
string comicInfoPath = Path.GetTempFileName(); string comicInfoPath = Path.GetTempFileName();
File.WriteAllText(comicInfoPath, GetComicInfoXmlString(publication, chapter, logger)); File.WriteAllText(comicInfoPath, chapter.GetComicInfoXmlString());
return DownloadChapterImages(imageUrls, GetArchiveFilePath(publication, chapter), (byte)1, parentTask, comicInfoPath, "https://chapmanganato.com/", cancellationToken); return DownloadChapterImages(imageUrls, chapter.GetArchiveFilePath(settings.downloadLocation), (byte)1, parentTask, comicInfoPath, "https://chapmanganato.com/", cancellationToken);
} }
private string[] ParseImageUrlsFromHtml(Stream html) private string[] ParseImageUrlsFromHtml(Stream html)

View File

@ -3,7 +3,6 @@ using System.Net;
using System.Text.RegularExpressions; using System.Text.RegularExpressions;
using System.Xml.Linq; using System.Xml.Linq;
using HtmlAgilityPack; using HtmlAgilityPack;
using Logging;
using Newtonsoft.Json; using Newtonsoft.Json;
using PuppeteerSharp; using PuppeteerSharp;
using Tranga.TrangaTasks; using Tranga.TrangaTasks;
@ -16,14 +15,13 @@ public class Mangasee : Connector
private IBrowser? _browser = null; private IBrowser? _browser = null;
private const string ChromiumVersion = "1154303"; private const string ChromiumVersion = "1154303";
public Mangasee(string downloadLocation, string imageCachePath, Logger? logger) : base(downloadLocation, public Mangasee(TrangaSettings settings) : base(settings)
imageCachePath, logger)
{ {
this.name = "Mangasee"; this.name = "Mangasee";
this.downloadClient = new DownloadClient(new Dictionary<byte, int>() this.downloadClient = new DownloadClient(new Dictionary<byte, int>()
{ {
{ (byte)1, 60 } { (byte)1, 60 }
}, logger); }, settings.logger);
Task d = new Task(DownloadBrowser); Task d = new Task(DownloadBrowser);
d.Start(); d.Start();
@ -36,31 +34,31 @@ public class Mangasee : Connector
browserFetcher.Remove(rev); browserFetcher.Remove(rev);
if (!browserFetcher.LocalRevisions().Contains(ChromiumVersion)) if (!browserFetcher.LocalRevisions().Contains(ChromiumVersion))
{ {
logger?.WriteLine(this.GetType().ToString(), "Downloading headless browser"); settings.logger?.WriteLine(this.GetType().ToString(), "Downloading headless browser");
DateTime last = DateTime.Now.Subtract(TimeSpan.FromSeconds(5)); DateTime last = DateTime.Now.Subtract(TimeSpan.FromSeconds(5));
browserFetcher.DownloadProgressChanged += (sender, args) => browserFetcher.DownloadProgressChanged += (sender, args) =>
{ {
double currentBytes = Convert.ToDouble(args.BytesReceived) / Convert.ToDouble(args.TotalBytesToReceive); double currentBytes = Convert.ToDouble(args.BytesReceived) / Convert.ToDouble(args.TotalBytesToReceive);
if (args.TotalBytesToReceive == args.BytesReceived) if (args.TotalBytesToReceive == args.BytesReceived)
{ {
logger?.WriteLine(this.GetType().ToString(), "Browser downloaded."); settings.logger?.WriteLine(this.GetType().ToString(), "Browser downloaded.");
} }
else if (DateTime.Now > last.AddSeconds(5)) else if (DateTime.Now > last.AddSeconds(5))
{ {
logger?.WriteLine(this.GetType().ToString(), $"Browser download progress: {currentBytes:P2}"); settings.logger?.WriteLine(this.GetType().ToString(), $"Browser download progress: {currentBytes:P2}");
last = DateTime.Now; last = DateTime.Now;
} }
}; };
if (!browserFetcher.CanDownloadAsync(ChromiumVersion).Result) if (!browserFetcher.CanDownloadAsync(ChromiumVersion).Result)
{ {
logger?.WriteLine(this.GetType().ToString(), $"Can't download browser version {ChromiumVersion}"); settings.logger?.WriteLine(this.GetType().ToString(), $"Can't download browser version {ChromiumVersion}");
return; return;
} }
await browserFetcher.DownloadAsync(ChromiumVersion); await browserFetcher.DownloadAsync(ChromiumVersion);
} }
logger?.WriteLine(this.GetType().ToString(), "Starting browser."); settings.logger?.WriteLine(this.GetType().ToString(), "Starting browser.");
this._browser = await Puppeteer.LaunchAsync(new LaunchOptions this._browser = await Puppeteer.LaunchAsync(new LaunchOptions
{ {
Headless = true, Headless = true,
@ -73,10 +71,9 @@ public class Mangasee : Connector
}); });
} }
public override Publication[] GetPublications(string publicationTitle = "") protected override Publication[] GetPublicationsInternal(string publicationTitle = "")
{ {
logger?.WriteLine(this.GetType().ToString(), $"Getting Publications (title={publicationTitle})"); settings.logger?.WriteLine(this.GetType().ToString(), $"Getting Publications (title={publicationTitle})");
string sanitizedTitle = string.Concat(Regex.Matches(publicationTitle, "[A-z]* *")).ToLower().Replace(' ', '+');
string requestUrl = $"https://mangasee123.com/_search.php"; string requestUrl = $"https://mangasee123.com/_search.php";
DownloadClient.RequestResult requestResult = DownloadClient.RequestResult requestResult =
downloadClient.MakeRequest(requestUrl, (byte)1); downloadClient.MakeRequest(requestUrl, (byte)1);
@ -93,26 +90,30 @@ public class Mangasee : Connector
Dictionary<SearchResultItem, int> queryFiltered = new(); Dictionary<SearchResultItem, int> queryFiltered = new();
foreach (SearchResultItem resultItem in result) foreach (SearchResultItem resultItem in result)
{ {
foreach (string term in publicationTitle.Split(' ')) int matches = resultItem.GetMatches(publicationTitle);
if (resultItem.i.Contains(term, StringComparison.CurrentCultureIgnoreCase)) if (matches > 0)
if (!queryFiltered.TryAdd(resultItem, 0)) queryFiltered.TryAdd(resultItem, matches);
queryFiltered[resultItem]++;
} }
queryFiltered = queryFiltered.Where(item => item.Value >= publicationTitle.Split(' ').Length - 1) queryFiltered = queryFiltered.Where(item => item.Value >= publicationTitle.Split(' ').Length - 1)
.ToDictionary(item => item.Key, item => item.Value); .ToDictionary(item => item.Key, item => item.Value);
settings.logger?.WriteLine(this.GetType().ToString(), $"Got {queryFiltered.Count} Publications (title={publicationTitle})");
HashSet<Publication> ret = new(); HashSet<Publication> ret = new();
List<SearchResultItem> orderedFiltered = List<SearchResultItem> orderedFiltered =
queryFiltered.OrderBy(item => item.Value).ToDictionary(item => item.Key, item => item.Value).Keys.ToList(); queryFiltered.OrderBy(item => item.Value).ToDictionary(item => item.Key, item => item.Value).Keys.ToList();
uint index = 1;
foreach (SearchResultItem orderedItem in orderedFiltered) foreach (SearchResultItem orderedItem in orderedFiltered)
{ {
DownloadClient.RequestResult requestResult = DownloadClient.RequestResult requestResult =
downloadClient.MakeRequest($"https://mangasee123.com/manga/{orderedItem.i}", (byte)1); downloadClient.MakeRequest($"https://mangasee123.com/manga/{orderedItem.i}", (byte)1);
if ((int)requestResult.statusCode < 200 || (int)requestResult.statusCode >= 300) if ((int)requestResult.statusCode >= 200 || (int)requestResult.statusCode < 300)
return Array.Empty<Publication>(); {
ret.Add(ParseSinglePublicationFromHtml(requestResult.result, orderedItem.s, orderedItem.i, orderedItem.a)); settings.logger?.WriteLine(this.GetType().ToString(), $"Retrieving Publication info: {orderedItem.s} {index++}/{orderedFiltered.Count}");
ret.Add(ParseSinglePublicationFromHtml(requestResult.result, orderedItem.s, orderedItem.i, orderedItem.a));
}
} }
return ret.ToArray(); return ret.ToArray();
} }
@ -121,9 +122,8 @@ public class Mangasee : Connector
private Publication ParseSinglePublicationFromHtml(Stream html, string sortName, string publicationId, string[] a) private Publication ParseSinglePublicationFromHtml(Stream html, string sortName, string publicationId, string[] a)
{ {
StreamReader reader = new (html); StreamReader reader = new (html);
string htmlString = reader.ReadToEnd();
HtmlDocument document = new (); HtmlDocument document = new ();
document.LoadHtml(htmlString); document.LoadHtml(reader.ReadToEnd());
string originalLanguage = "", status = ""; string originalLanguage = "", status = "";
Dictionary<string, string> altTitles = new(), links = new(); Dictionary<string, string> altTitles = new(), links = new();
@ -182,6 +182,30 @@ public class Mangasee : Connector
public string s { get; set; } public string s { get; set; }
public string[] a { get; set; } public string[] a { get; set; }
#pragma warning restore CS8618 #pragma warning restore CS8618
public int GetMatches(string title)
{
int ret = 0;
Regex cleanRex = new("[A-z0-9]*");
string[] badWords = { "a", "an", "no", "ni", "so", "as", "and", "the", "of", "that", "in", "is", "for" };
string[] titleTerms = title.Split(new[] { ' ', '-' }).Where(str => !badWords.Contains(str)).ToArray();
foreach (Match matchTerm in cleanRex.Matches(this.i))
ret += titleTerms.Count(titleTerm =>
titleTerm.Equals(matchTerm.Value, StringComparison.OrdinalIgnoreCase));
foreach (Match matchTerm in cleanRex.Matches(this.s))
ret += titleTerms.Count(titleTerm =>
titleTerm.Equals(matchTerm.Value, StringComparison.OrdinalIgnoreCase));
foreach(string alt in this.a)
foreach (Match matchTerm in cleanRex.Matches(alt))
ret += titleTerms.Count(titleTerm =>
titleTerm.Equals(matchTerm.Value, StringComparison.OrdinalIgnoreCase));
return ret;
}
} }
public override Chapter[] GetChapters(Publication publication, string language = "") public override Chapter[] GetChapters(Publication publication, string language = "")
@ -197,7 +221,7 @@ public class Mangasee : Connector
string url = chapter.Descendants("link").First().Value; string url = chapter.Descendants("link").First().Value;
url = url.Replace(Regex.Matches(url,"(-page-[0-9])")[0].ToString(),""); url = url.Replace(Regex.Matches(url,"(-page-[0-9])")[0].ToString(),"");
ret.Add(new Chapter("", volumeNumber, chapterNumber, url)); ret.Add(new Chapter(publication, "", volumeNumber, chapterNumber, url));
} }
//Return Chapters ordered by Chapter-Number //Return Chapters ordered by Chapter-Number
@ -205,7 +229,7 @@ public class Mangasee : Connector
{ {
NumberDecimalSeparator = "." NumberDecimalSeparator = "."
}; };
logger?.WriteLine(this.GetType().ToString(), $"Done getting Chapters for {publication.internalId}"); settings.logger?.WriteLine(this.GetType().ToString(), $"Done getting Chapters for {publication.internalId}");
return ret.OrderBy(chapter => Convert.ToSingle(chapter.chapterNumber, chapterNumberFormatInfo)).ToArray(); return ret.OrderBy(chapter => Convert.ToSingle(chapter.chapterNumber, chapterNumberFormatInfo)).ToArray();
} }
@ -215,13 +239,13 @@ public class Mangasee : Connector
return HttpStatusCode.RequestTimeout; return HttpStatusCode.RequestTimeout;
while (this._browser is null && !(cancellationToken?.IsCancellationRequested??false)) while (this._browser is null && !(cancellationToken?.IsCancellationRequested??false))
{ {
logger?.WriteLine(this.GetType().ToString(), "Waiting for headless browser to download..."); settings.logger?.WriteLine(this.GetType().ToString(), "Waiting for headless browser to download...");
Thread.Sleep(1000); Thread.Sleep(1000);
} }
if (cancellationToken?.IsCancellationRequested??false) if (cancellationToken?.IsCancellationRequested??false)
return HttpStatusCode.RequestTimeout; return HttpStatusCode.RequestTimeout;
logger?.WriteLine(this.GetType().ToString(), $"Downloading Chapter-Info {publication.sortName} {publication.internalId} {chapter.volumeNumber}-{chapter.chapterNumber}"); settings.logger?.WriteLine(this.GetType().ToString(), $"Downloading Chapter-Info {publication.sortName} {publication.internalId} {chapter.volumeNumber}-{chapter.chapterNumber}");
IPage page = _browser!.NewPageAsync().Result; IPage page = _browser!.NewPageAsync().Result;
IResponse response = page.GoToAsync(chapter.url).Result; IResponse response = page.GoToAsync(chapter.url).Result;
if (response.Ok) if (response.Ok)
@ -236,9 +260,9 @@ public class Mangasee : Connector
urls.Add(galleryImage.GetAttributeValue("src", "")); urls.Add(galleryImage.GetAttributeValue("src", ""));
string comicInfoPath = Path.GetTempFileName(); string comicInfoPath = Path.GetTempFileName();
File.WriteAllText(comicInfoPath, GetComicInfoXmlString(publication, chapter, logger)); File.WriteAllText(comicInfoPath, chapter.GetComicInfoXmlString());
return DownloadChapterImages(urls.ToArray(), GetArchiveFilePath(publication, chapter), (byte)1, parentTask, comicInfoPath, cancellationToken:cancellationToken); return DownloadChapterImages(urls.ToArray(), chapter.GetArchiveFilePath(settings.downloadLocation), (byte)1, parentTask, comicInfoPath, cancellationToken:cancellationToken);
} }
return response.Status; return response.Status;
} }

108
Tranga/DownloadClient.cs Normal file
View File

@ -0,0 +1,108 @@
using System.Net;
using Logging;
namespace Tranga;
internal class DownloadClient
{
private static readonly HttpClient Client = new()
{
Timeout = TimeSpan.FromSeconds(60)
};
private readonly Dictionary<byte, DateTime> _lastExecutedRateLimit;
private readonly Dictionary<byte, TimeSpan> _rateLimit;
// ReSharper disable once InconsistentNaming
private readonly Logger? logger;
/// <summary>
/// Creates a httpClient
/// </summary>
/// <param name="rateLimitRequestsPerMinute">Rate limits for requests. byte is RequestType, int maximum requests per minute for RequestType</param>
/// <param name="logger"></param>
public DownloadClient(Dictionary<byte, int> rateLimitRequestsPerMinute, Logger? logger)
{
this.logger = logger;
_lastExecutedRateLimit = new();
_rateLimit = new();
foreach(KeyValuePair<byte, int> limit in rateLimitRequestsPerMinute)
_rateLimit.Add(limit.Key, TimeSpan.FromMinutes(1).Divide(limit.Value));
}
/// <summary>
/// Request Webpage
/// </summary>
/// <param name="url"></param>
/// <param name="requestType">For RateLimits: Same Endpoints use same type</param>
/// <param name="referrer">Used in http request header</param>
/// <returns>RequestResult with StatusCode and Stream of received data</returns>
public RequestResult MakeRequest(string url, byte requestType, string? referrer = null)
{
if (_rateLimit.TryGetValue(requestType, out TimeSpan value))
_lastExecutedRateLimit.TryAdd(requestType, DateTime.Now.Subtract(value));
else
{
logger?.WriteLine(this.GetType().ToString(), "RequestType not configured for rate-limit.");
return new RequestResult(HttpStatusCode.NotAcceptable, Stream.Null);
}
TimeSpan rateLimitTimeout = _rateLimit[requestType]
.Subtract(DateTime.Now.Subtract(_lastExecutedRateLimit[requestType]));
if(rateLimitTimeout > TimeSpan.Zero)
Thread.Sleep(rateLimitTimeout);
HttpResponseMessage? response = null;
while (response is null)
{
try
{
HttpRequestMessage requestMessage = new(HttpMethod.Get, url);
if(referrer is not null)
requestMessage.Headers.Referrer = new Uri(referrer);
_lastExecutedRateLimit[requestType] = DateTime.Now;
response = Client.Send(requestMessage);
}
catch (HttpRequestException e)
{
logger?.WriteLine(this.GetType().ToString(), e.Message);
logger?.WriteLine(this.GetType().ToString(), $"Waiting {_rateLimit[requestType] * 2}... Retrying.");
Thread.Sleep(_rateLimit[requestType] * 2);
}
}
if (!response.IsSuccessStatusCode)
{
logger?.WriteLine(this.GetType().ToString(), $"Request-Error {response.StatusCode}: {response.ReasonPhrase}");
return new RequestResult(response.StatusCode, Stream.Null);
}
// Request has been redirected to another page. For example, it redirects directly to the results when there is only 1 result
if(response.RequestMessage is not null && response.RequestMessage.RequestUri is not null)
{
return new RequestResult(response.StatusCode, response.Content.ReadAsStream(), true, response.RequestMessage.RequestUri.AbsoluteUri);
}
return new RequestResult(response.StatusCode, response.Content.ReadAsStream());
}
public struct RequestResult
{
public HttpStatusCode statusCode { get; }
public Stream result { get; }
public bool HasBeenRedirected { get; }
public string? RedirectedToUrl { get; }
public RequestResult(HttpStatusCode statusCode, Stream result)
{
this.statusCode = statusCode;
this.result = result;
}
public RequestResult(HttpStatusCode statusCode, Stream result, bool hasBeenRedirected, string redirectedTo)
: this(statusCode, result)
{
this.HasBeenRedirected = hasBeenRedirected;
RedirectedToUrl = redirectedTo;
}
}
}

View File

@ -5,7 +5,7 @@ using Newtonsoft.Json;
using Newtonsoft.Json.Linq; using Newtonsoft.Json.Linq;
using Tranga.LibraryManagers; using Tranga.LibraryManagers;
namespace Tranga; namespace Tranga.LibraryManagers;
public abstract class LibraryManager public abstract class LibraryManager
{ {

34
Tranga/Migrate.cs Normal file
View File

@ -0,0 +1,34 @@
using Logging;
using Newtonsoft.Json;
using Tranga.TrangaTasks;
namespace Tranga;
public static class Migrate
{
private static readonly ushort CurrentVersion = 16;
public static void Files(TrangaSettings settings)
{
settings.version ??= 15;
switch (settings.version)
{
case 15:
RemoveUpdateLibraryTask(settings);
break;
}
settings.version = CurrentVersion;
settings.ExportSettings();
}
private static void RemoveUpdateLibraryTask(TrangaSettings settings)
{
if (!File.Exists(settings.tasksFilePath))
return;
string tasksJsonString = File.ReadAllText(settings.tasksFilePath);
HashSet<TrangaTask> tasks = JsonConvert.DeserializeObject<HashSet<TrangaTask>>(tasksJsonString, new JsonSerializerSettings { Converters = { new TrangaTask.TrangaTaskJsonConverter() } })!;
tasks.RemoveWhere(t => t.task == TrangaTask.Task.UpdateLibraries);
File.WriteAllText(settings.tasksFilePath, JsonConvert.SerializeObject(tasks));
}
}

View File

@ -6,18 +6,18 @@ namespace Tranga.NotificationManagers;
public class LunaSea : NotificationManager public class LunaSea : NotificationManager
{ {
public string webhook { get; } public string id { get; }
private readonly HttpClient _client = new(); private readonly HttpClient _client = new();
public LunaSea(string webhook, Logger? logger = null) : base(NotificationManagerType.LunaSea, logger) public LunaSea(string id, Logger? logger = null) : base(NotificationManagerType.LunaSea, logger)
{ {
this.webhook = webhook; this.id = id;
} }
public override void SendNotification(string title, string notificationText) public override void SendNotification(string title, string notificationText)
{ {
logger?.WriteLine(this.GetType().ToString(), $"Sending notification: {title} - {notificationText}"); logger?.WriteLine(this.GetType().ToString(), $"Sending notification: {title} - {notificationText}");
MessageData message = new(title, notificationText); MessageData message = new(title, notificationText);
HttpRequestMessage request = new(HttpMethod.Post, webhook); HttpRequestMessage request = new(HttpMethod.Post, $"https://notify.lunasea.app/v1/custom/{id}");
request.Content = new StringContent(JsonConvert.SerializeObject(message, Formatting.None), Encoding.UTF8, "application/json"); request.Content = new StringContent(JsonConvert.SerializeObject(message, Formatting.None), Encoding.UTF8, "application/json");
HttpResponseMessage response = _client.Send(request); HttpResponseMessage response = _client.Send(request);
if (!response.IsSuccessStatusCode) if (!response.IsSuccessStatusCode)
@ -31,13 +31,11 @@ public class LunaSea : NotificationManager
{ {
public string title { get; } public string title { get; }
public string body { get; } public string body { get; }
public string image { get; }
public MessageData(string title, string body) public MessageData(string title, string body)
{ {
this.title = title; this.title = title;
this.body = body; this.body = body;
this.image = "";
} }
} }
} }

View File

@ -1,13 +1,12 @@
using Logging; using Logging;
using Newtonsoft.Json; using Newtonsoft.Json;
using Newtonsoft.Json.Linq; using Newtonsoft.Json.Linq;
using Tranga.NotificationManagers;
namespace Tranga; namespace Tranga.NotificationManagers;
public abstract class NotificationManager public abstract class NotificationManager
{ {
protected readonly Logger? logger; protected Logger? logger;
public NotificationManagerType notificationManagerType; public NotificationManagerType notificationManagerType;
protected NotificationManager(NotificationManagerType notificationManagerType, Logger? logger = null) protected NotificationManager(NotificationManagerType notificationManagerType, Logger? logger = null)
@ -19,6 +18,11 @@ public abstract class NotificationManager
public enum NotificationManagerType : byte { Gotify = 0, LunaSea = 1 } public enum NotificationManagerType : byte { Gotify = 0, LunaSea = 1 }
public abstract void SendNotification(string title, string notificationText); public abstract void SendNotification(string title, string notificationText);
public void AddLogger(Logger pLogger)
{
this.logger = pLogger;
}
public class NotificationManagerJsonConverter : JsonConverter public class NotificationManagerJsonConverter : JsonConverter
{ {

View File

@ -9,7 +9,7 @@ namespace Tranga;
/// <summary> /// <summary>
/// Contains information on a Publication (Manga) /// Contains information on a Publication (Manga)
/// </summary> /// </summary>
public readonly struct Publication public struct Publication
{ {
public string sortName { get; } public string sortName { get; }
public List<string> authors { get; } public List<string> authors { get; }
@ -26,22 +26,12 @@ public readonly struct Publication
public string folderName { get; } public string folderName { get; }
public string publicationId { get; } public string publicationId { get; }
public string internalId { get; } public string internalId { get; }
public float ignoreChaptersBelow { get; set; }
private static readonly Regex LegalCharacters = new Regex(@"[A-Z]*[a-z]*[0-9]* *\.*-*,*'*\'*\)*\(*~*!*"); private static readonly Regex LegalCharacters = new Regex(@"[A-Z]*[a-z]*[0-9]* *\.*-*,*'*\'*\)*\(*~*!*");
[JsonConstructor] //Legacy [JsonConstructor]
public Publication(string sortName, string? author, string? description, Dictionary<string, string> altTitles, public Publication(string sortName, List<string> authors, string? description, Dictionary<string,string> altTitles, string[] tags, string? posterUrl, string? coverFileNameInCache, Dictionary<string,string>? links, int? year, string? originalLanguage, string status, string publicationId, string? folderName = null, float? ignoreChaptersBelow = 0)
string[] tags, string? posterUrl, string? coverFileNameInCache, Dictionary<string, string>? links, int? year,
string? originalLanguage, string status, string publicationId)
{
List<string> pAuthors = new();
if(author is not null)
pAuthors.Add(author);
this = new Publication(sortName, pAuthors, description, altTitles, tags, posterUrl,
coverFileNameInCache, links, year, originalLanguage, status, publicationId);
}
public Publication(string sortName, List<string> authors, string? description, Dictionary<string,string> altTitles, string[] tags, string? posterUrl, string? coverFileNameInCache, Dictionary<string,string>? links, int? year, string? originalLanguage, string status, string publicationId)
{ {
this.sortName = sortName; this.sortName = sortName;
this.authors = authors; this.authors = authors;
@ -55,11 +45,12 @@ public readonly struct Publication
this.originalLanguage = originalLanguage; this.originalLanguage = originalLanguage;
this.status = status; this.status = status;
this.publicationId = publicationId; this.publicationId = publicationId;
this.folderName = string.Concat(LegalCharacters.Matches(sortName)); this.folderName = folderName ?? string.Concat(LegalCharacters.Matches(sortName));
while (this.folderName.EndsWith('.')) while (this.folderName.EndsWith('.'))
this.folderName = this.folderName.Substring(0, this.folderName.Length - 1); this.folderName = this.folderName.Substring(0, this.folderName.Length - 1);
string onlyLowerLetters = string.Concat(this.sortName.ToLower().Where(Char.IsLetter)); string onlyLowerLetters = string.Concat(this.sortName.ToLower().Where(Char.IsLetter));
this.internalId = Convert.ToBase64String(Encoding.ASCII.GetBytes($"{onlyLowerLetters}{this.year}")); this.internalId = Convert.ToBase64String(Encoding.ASCII.GetBytes($"{onlyLowerLetters}{this.year}"));
this.ignoreChaptersBelow = ignoreChaptersBelow ?? 0f;
} }
public string CreatePublicationFolder(string downloadDirectory) public string CreatePublicationFolder(string downloadDirectory)

View File

@ -11,26 +11,31 @@ namespace Tranga;
/// </summary> /// </summary>
public class TaskManager public class TaskManager
{ {
public Dictionary<Publication, List<Chapter>> chapterCollection = new(); public HashSet<Publication> collection = new();
private HashSet<TrangaTask> _allTasks = new(); private HashSet<TrangaTask> _allTasks = new();
private readonly Dictionary<TrangaTask, CancellationTokenSource> _runningTasks = new ();
private bool _continueRunning = true; private bool _continueRunning = true;
private readonly Connector[] _connectors; private readonly Connector[] _connectors;
public TrangaSettings settings { get; } public TrangaSettings settings { get; }
private Logger? logger { get; }
private readonly Dictionary<DownloadChapterTask, CancellationTokenSource> _runningDownloadChapterTasks = new(); public TaskManager(TrangaSettings settings)
public TaskManager(TrangaSettings settings, Logger? logger = null)
{ {
this.logger = logger; settings.logger?.WriteLine("Tranga", value: "\n"+
@"-----------------------------------------------------------------"+"\n"+
@" |¯¯¯¯¯¯|°|¯¯¯¯¯¯\ /¯¯¯¯¯¯| |¯¯¯\|¯¯¯| /¯¯¯¯¯¯\' /¯¯¯¯¯¯| "+"\n"+
@" | | | x <|' / ! | | '| | (/¯¯¯\° / ! | "+ "\n"+
@" ¯|__|¯ |__|\\__\\ /___/¯|_'| |___|\\__| \\_____/' /___/¯|_'| "+ "\n"+
@"-----------------------------------------------------------------");
this._connectors = new Connector[] this._connectors = new Connector[]
{ {
new MangaDex(settings.downloadLocation, settings.coverImageCache, logger), new MangaDex(settings),
new Manganato(settings.downloadLocation, settings.coverImageCache, logger), new Manganato(settings),
new Mangasee(settings.downloadLocation, settings.coverImageCache, logger) new Mangasee(settings),
new MangaKatana(settings)
}; };
this.settings = settings; this.settings = settings;
Migrate.Files(settings);
ImportData(); ImportData();
ExportDataAndSettings(); ExportDataAndSettings();
Thread taskChecker = new(TaskCheckerThread); Thread taskChecker = new(TaskCheckerThread);
@ -43,7 +48,7 @@ public class TaskManager
/// </summary> /// </summary>
private void TaskCheckerThread() private void TaskCheckerThread()
{ {
logger?.WriteLine(this.GetType().ToString(), "Starting TaskCheckerThread."); settings.logger?.WriteLine(this.GetType().ToString(), "Starting TaskCheckerThread.");
int waitingTasksCount = _allTasks.Count(task => task.state is TrangaTask.ExecutionState.Waiting); int waitingTasksCount = _allTasks.Count(task => task.state is TrangaTask.ExecutionState.Waiting);
while (_continueRunning) while (_continueRunning)
{ {
@ -54,7 +59,7 @@ public class TaskManager
waitingButExecute.state = TrangaTask.ExecutionState.Enqueued; waitingButExecute.state = TrangaTask.ExecutionState.Enqueued;
} }
foreach (TrangaTask enqueuedTask in _allTasks.Where(enqueuedTask => enqueuedTask.state is TrangaTask.ExecutionState.Enqueued)) foreach (TrangaTask enqueuedTask in _allTasks.Where(enqueuedTask => enqueuedTask.state is TrangaTask.ExecutionState.Enqueued).OrderBy(enqueuedTask => enqueuedTask.nextExecution))
{ {
switch (enqueuedTask.task) switch (enqueuedTask.task)
{ {
@ -82,21 +87,35 @@ public class TaskManager
} }
} }
TrangaTask[] failedDownloadChapterTasks = _allTasks.Where(taskQuery => foreach (TrangaTask timedOutTask in _runningTasks.Keys
taskQuery.state is TrangaTask.ExecutionState.Failed && taskQuery is DownloadChapterTask).ToArray(); .Where(taskQuery => taskQuery.lastChange < DateTime.Now.Subtract(TimeSpan.FromMinutes(3))))
foreach (TrangaTask failedDownloadChapterTask in failedDownloadChapterTasks)
{ {
DeleteTask(failedDownloadChapterTask); _runningTasks[timedOutTask].Cancel();
TrangaTask newTask = failedDownloadChapterTask.Clone(); timedOutTask.state = TrangaTask.ExecutionState.Failed;
failedDownloadChapterTask.parentTask?.AddChildTask(newTask);
AddTask(newTask);
} }
TrangaTask[] successfulDownloadChapterTasks = _allTasks.Where(taskQuery => foreach (TrangaTask finishedTask in _allTasks
taskQuery.state is TrangaTask.ExecutionState.Success && taskQuery is DownloadChapterTask).ToArray(); .Where(taskQuery => taskQuery.state is TrangaTask.ExecutionState.Success).ToArray())
foreach(TrangaTask successfulDownloadChapterTask in successfulDownloadChapterTasks)
{ {
DeleteTask(successfulDownloadChapterTask); if(finishedTask is DownloadChapterTask)
{
DeleteTask(finishedTask);
finishedTask.state = TrangaTask.ExecutionState.Success;
}
else
{
finishedTask.state = TrangaTask.ExecutionState.Waiting;
this._runningTasks.Remove(finishedTask);
}
}
foreach (TrangaTask failedTask in _allTasks.Where(taskQuery =>
taskQuery.state is TrangaTask.ExecutionState.Failed).ToArray())
{
DeleteTask(failedTask);
TrangaTask newTask = failedTask.Clone();
failedTask.parentTask?.AddChildTask(newTask);
AddTask(newTask);
} }
if(waitingTasksCount != _allTasks.Count(task => task.state is TrangaTask.ExecutionState.Waiting)) if(waitingTasksCount != _allTasks.Count(task => task.state is TrangaTask.ExecutionState.Waiting))
@ -116,59 +135,72 @@ public class TaskManager
CancellationTokenSource cToken = new (); CancellationTokenSource cToken = new ();
Task t = new(() => Task t = new(() =>
{ {
task.Execute(this, this.logger, cToken.Token); task.Execute(this, cToken.Token);
}, cToken.Token); }, cToken.Token);
if(task is DownloadChapterTask chapterTask) _runningTasks.Add(task, cToken);
_runningDownloadChapterTasks.Add(chapterTask, cToken);
t.Start(); t.Start();
} }
public void AddTask(TrangaTask newTask) public void AddTask(TrangaTask newTask)
{ {
logger?.WriteLine(this.GetType().ToString(), $"Adding new Task {newTask}");
switch (newTask.task) switch (newTask.task)
{ {
case TrangaTask.Task.UpdateLibraries: case TrangaTask.Task.UpdateLibraries:
//Only one UpdateKomgaLibrary Task //Only one UpdateKomgaLibrary Task
logger?.WriteLine(this.GetType().ToString(), $"Replacing old {newTask.task}-Task."); settings.logger?.WriteLine(this.GetType().ToString(), $"Replacing old {newTask.task}-Task.");
_allTasks.RemoveWhere(trangaTask => trangaTask.task is TrangaTask.Task.UpdateLibraries); if (GetTasksMatching(newTask).FirstOrDefault() is { } exists)
_allTasks.Remove(exists);
_allTasks.Add(newTask); _allTasks.Add(newTask);
ExportDataAndSettings();
break; break;
case TrangaTask.Task.MonitorPublication: default:
if (!_allTasks.Any(mTask => mTask is MonitorPublicationTask mpt && newTask is MonitorPublicationTask nMpt && if (!GetTasksMatching(newTask).Any())
mpt.publication.internalId == nMpt.publication.internalId && {
mpt.connectorName == nMpt.connectorName)) settings.logger?.WriteLine(this.GetType().ToString(), $"Adding new Task {newTask}");
_allTasks.Add(newTask); _allTasks.Add(newTask);
ExportDataAndSettings();
}
else else
logger?.WriteLine(this.GetType().ToString(), $"Task already exists {newTask}"); settings.logger?.WriteLine(this.GetType().ToString(), $"Task already exists {newTask}");
break;
case TrangaTask.Task.DownloadChapter:
if (!_allTasks.Any(mTask => mTask is DownloadChapterTask dct && newTask is DownloadChapterTask nDct &&
dct.publication.internalId == nDct.publication.internalId &&
dct.connectorName == nDct.connectorName &&
dct.chapter.sortNumber == nDct.chapter.sortNumber))
_allTasks.Add(newTask);
else
logger?.WriteLine(this.GetType().ToString(), $"Task already exists {newTask}");
break; break;
} }
ExportDataAndSettings();
} }
public void DeleteTask(TrangaTask removeTask) public void DeleteTask(TrangaTask removeTask)
{ {
logger?.WriteLine(this.GetType().ToString(), $"Removing Task {removeTask}"); settings.logger?.WriteLine(this.GetType().ToString(), $"Removing Task {removeTask}");
_allTasks.Remove(removeTask); if(_allTasks.Contains(removeTask))
_allTasks.Remove(removeTask);
removeTask.parentTask?.RemoveChildTask(removeTask); removeTask.parentTask?.RemoveChildTask(removeTask);
if (removeTask is DownloadChapterTask cRemoveTask && _runningDownloadChapterTasks.ContainsKey(cRemoveTask)) if (_runningTasks.ContainsKey(removeTask))
{ {
_runningDownloadChapterTasks[cRemoveTask].Cancel(); _runningTasks[removeTask].Cancel();
_runningDownloadChapterTasks.Remove(cRemoveTask); _runningTasks.Remove(removeTask);
} }
foreach(TrangaTask childTask in removeTask.childTasks)
DeleteTask(childTask);
ExportDataAndSettings();
} }
public IEnumerable<TrangaTask> GetTasksMatching(TrangaTask.Task taskType, string? connectorName = null, string? searchString = null, string? internalId = null, string? chapterSortNumber = null) public IEnumerable<TrangaTask> GetTasksMatching(TrangaTask mTask)
{
switch (mTask.task)
{
case TrangaTask.Task.UpdateLibraries:
return GetTasksMatching(TrangaTask.Task.UpdateLibraries);
case TrangaTask.Task.DownloadChapter:
DownloadChapterTask dct = (DownloadChapterTask)mTask;
return GetTasksMatching(TrangaTask.Task.DownloadChapter, connectorName: dct.connectorName,
internalId: dct.publication.internalId, chapterNumber: dct.chapter.chapterNumber);
case TrangaTask.Task.MonitorPublication:
MonitorPublicationTask mpt = (MonitorPublicationTask)mTask;
return GetTasksMatching(TrangaTask.Task.MonitorPublication, connectorName: mpt.connectorName,
internalId: mpt.publication.internalId);
}
return Array.Empty<TrangaTask>();
}
public IEnumerable<TrangaTask> GetTasksMatching(TrangaTask.Task taskType, string? connectorName = null, string? searchString = null, string? internalId = null, string? chapterNumber = null)
{ {
switch (taskType) switch (taskType)
{ {
@ -204,12 +236,12 @@ public class TaskManager
mTask is DownloadChapterTask dct && dct.connectorName == connectorName && mTask is DownloadChapterTask dct && dct.connectorName == connectorName &&
dct.ToString().Contains(searchString, StringComparison.InvariantCultureIgnoreCase)); dct.ToString().Contains(searchString, StringComparison.InvariantCultureIgnoreCase));
} }
else if (internalId is not null && chapterSortNumber is not null) else if (internalId is not null && chapterNumber is not null)
{ {
return _allTasks.Where(mTask => return _allTasks.Where(mTask =>
mTask is DownloadChapterTask dct && dct.connectorName == connectorName && mTask is DownloadChapterTask dct && dct.connectorName == connectorName &&
dct.publication.publicationId == internalId && dct.publication.internalId == internalId &&
dct.chapter.sortNumber == chapterSortNumber); dct.chapter.chapterNumber == chapterNumber);
} }
else else
return _allTasks.Where(mTask => return _allTasks.Where(mTask =>
@ -253,47 +285,17 @@ public class TaskManager
_allTasks.CopyTo(ret); _allTasks.CopyTo(ret);
return ret; return ret;
} }
public Publication[] GetPublicationsFromConnector(Connector connector, string? title = null)
{
Publication[] ret = connector.GetPublications(title ?? "");
foreach (Publication publication in ret)
{
if(chapterCollection.All(pub => pub.Key.internalId != publication.internalId))
this.chapterCollection.TryAdd(publication, new List<Chapter>());
}
return ret;
}
/// <returns>All added Publications</returns> /// <returns>All added Publications</returns>
public Publication[] GetAllPublications() public Publication[] GetAllPublications()
{ {
return this.chapterCollection.Keys.ToArray(); return this.collection.ToArray();
}
/// <summary>
/// Updates the available Chapters of a Publication
/// </summary>
/// <param name="connector">Connector to use</param>
/// <param name="publication">Publication to check</param>
/// <param name="language">Language to receive chapters for</param>
/// <returns>List of Chapters that were previously not in collection</returns>
public List<Chapter> GetNewChaptersList(Connector connector, Publication publication, string language)
{
List<Chapter> newChaptersList = new();
chapterCollection.TryAdd(publication, newChaptersList); //To ensure publication is actually in collection
Chapter[] newChapters = connector.GetChapters(publication, language);
newChaptersList = newChapters.Where(nChapter => !connector.CheckChapterIsDownloaded(publication, nChapter)).ToList();
return newChaptersList;
} }
public List<Chapter> GetExistingChaptersList(Connector connector, Publication publication, string language) public List<Chapter> GetExistingChaptersList(Connector connector, Publication publication, string language)
{ {
Chapter[] newChapters = connector.GetChapters(publication, language); Chapter[] newChapters = connector.GetChapters(publication, language);
return newChapters.Where(nChapter => connector.CheckChapterIsDownloaded(publication, nChapter)).ToList(); return newChapters.Where(nChapter => nChapter.CheckChapterIsDownloaded(settings.downloadLocation)).ToList();
} }
/// <summary> /// <summary>
@ -317,7 +319,7 @@ public class TaskManager
/// <param name="force">If force is true, tasks are aborted.</param> /// <param name="force">If force is true, tasks are aborted.</param>
public void Shutdown(bool force = false) public void Shutdown(bool force = false)
{ {
logger?.WriteLine(this.GetType().ToString(), $"Shutting down (forced={force})"); settings.logger?.WriteLine(this.GetType().ToString(), $"Shutting down (forced={force})");
_continueRunning = false; _continueRunning = false;
ExportDataAndSettings(); ExportDataAndSettings();
@ -327,40 +329,30 @@ public class TaskManager
//Wait for tasks to finish //Wait for tasks to finish
while(_allTasks.Any(task => task.state is TrangaTask.ExecutionState.Running or TrangaTask.ExecutionState.Enqueued)) while(_allTasks.Any(task => task.state is TrangaTask.ExecutionState.Running or TrangaTask.ExecutionState.Enqueued))
Thread.Sleep(10); Thread.Sleep(10);
logger?.WriteLine(this.GetType().ToString(), "Tasks finished. Bye!"); settings.logger?.WriteLine(this.GetType().ToString(), "Tasks finished. Bye!");
Environment.Exit(0); Environment.Exit(0);
} }
private void ImportData() private void ImportData()
{ {
logger?.WriteLine(this.GetType().ToString(), "Importing Data"); settings.logger?.WriteLine(this.GetType().ToString(), "Importing Data");
string buffer; string buffer;
if (File.Exists(settings.tasksFilePath)) if (File.Exists(settings.tasksFilePath))
{ {
logger?.WriteLine(this.GetType().ToString(), $"Importing tasks from {settings.tasksFilePath}"); settings.logger?.WriteLine(this.GetType().ToString(), $"Importing tasks from {settings.tasksFilePath}");
buffer = File.ReadAllText(settings.tasksFilePath); buffer = File.ReadAllText(settings.tasksFilePath);
this._allTasks = JsonConvert.DeserializeObject<HashSet<TrangaTask>>(buffer, new JsonSerializerSettings() { Converters = { new TrangaTask.TrangaTaskJsonConverter() } })!; this._allTasks = JsonConvert.DeserializeObject<HashSet<TrangaTask>>(buffer, new JsonSerializerSettings() { Converters = { new TrangaTask.TrangaTaskJsonConverter() } })!;
} }
foreach (TrangaTask task in this._allTasks.Where(tTask => tTask.parentTaskId is not null)) foreach (TrangaTask task in this._allTasks.Where(tTask => tTask.parentTaskId is not null).ToArray())
{ {
TrangaTask? parentTask = this._allTasks.FirstOrDefault(pTask => pTask.taskId == task.parentTaskId); TrangaTask? parentTask = this._allTasks.FirstOrDefault(pTask => pTask.taskId == task.parentTaskId);
if (parentTask is not null) if (parentTask is not null)
{ {
task.parentTask = parentTask; this.DeleteTask(task);
parentTask.AddChildTask(task); parentTask.lastExecuted = DateTime.UnixEpoch;
} }
} }
if (File.Exists(settings.knownPublicationsPath))
{
logger?.WriteLine(this.GetType().ToString(), $"Importing known publications from {settings.knownPublicationsPath}");
buffer = File.ReadAllText(settings.knownPublicationsPath);
Publication[] publications = JsonConvert.DeserializeObject<Publication[]>(buffer)!;
foreach (Publication publication in publications)
this.chapterCollection.TryAdd(publication, new List<Chapter>());
}
} }
/// <summary> /// <summary>
@ -368,20 +360,13 @@ public class TaskManager
/// </summary> /// </summary>
private void ExportDataAndSettings() private void ExportDataAndSettings()
{ {
logger?.WriteLine(this.GetType().ToString(), $"Exporting settings to {settings.settingsFilePath}"); settings.logger?.WriteLine(this.GetType().ToString(), $"Exporting settings to {settings.settingsFilePath}");
while(IsFileInUse(settings.settingsFilePath)) settings.ExportSettings();
Thread.Sleep(50);
File.WriteAllText(settings.settingsFilePath, JsonConvert.SerializeObject(settings));
logger?.WriteLine(this.GetType().ToString(), $"Exporting tasks to {settings.tasksFilePath}"); settings.logger?.WriteLine(this.GetType().ToString(), $"Exporting tasks to {settings.tasksFilePath}");
while(IsFileInUse(settings.tasksFilePath)) while(IsFileInUse(settings.tasksFilePath))
Thread.Sleep(50); Thread.Sleep(50);
File.WriteAllText(settings.tasksFilePath, JsonConvert.SerializeObject(this._allTasks)); File.WriteAllText(settings.tasksFilePath, JsonConvert.SerializeObject(this._allTasks));
logger?.WriteLine(this.GetType().ToString(), $"Exporting known publications to {settings.knownPublicationsPath}");
while(IsFileInUse(settings.knownPublicationsPath))
Thread.Sleep(50);
File.WriteAllText(settings.knownPublicationsPath, JsonConvert.SerializeObject(this.chapterCollection.Keys.ToArray()));
} }
private bool IsFileInUse(string path) private bool IsFileInUse(string path)

View File

@ -7,17 +7,18 @@ namespace Tranga;
public class TrangaSettings public class TrangaSettings
{ {
public string downloadLocation { get; set; } public string downloadLocation { get; private set; }
public string workingDirectory { get; set; } public string workingDirectory { get; init; }
[JsonIgnore] public string settingsFilePath => Path.Join(workingDirectory, "settings.json"); [JsonIgnore] public string settingsFilePath => Path.Join(workingDirectory, "settings.json");
[JsonIgnore] public string tasksFilePath => Path.Join(workingDirectory, "tasks.json"); [JsonIgnore] public string tasksFilePath => Path.Join(workingDirectory, "tasks.json");
[JsonIgnore] public string knownPublicationsPath => Path.Join(workingDirectory, "knownPublications.json");
[JsonIgnore] public string coverImageCache => Path.Join(workingDirectory, "imageCache"); [JsonIgnore] public string coverImageCache => Path.Join(workingDirectory, "imageCache");
public HashSet<LibraryManager> libraryManagers { get; } public HashSet<LibraryManager> libraryManagers { get; }
public HashSet<NotificationManager> notificationManagers { get; } public HashSet<NotificationManager> notificationManagers { get; }
public ushort? version { get; set; }
[JsonIgnore] public Logger? logger { get; init; }
public TrangaSettings(string downloadLocation, string workingDirectory, HashSet<LibraryManager>? libraryManagers, public TrangaSettings(string downloadLocation, string workingDirectory, HashSet<LibraryManager>? libraryManagers,
HashSet<NotificationManager>? notificationManagers) HashSet<NotificationManager>? notificationManagers, Logger? logger)
{ {
if (downloadLocation.Length < 1 || workingDirectory.Length < 1) if (downloadLocation.Length < 1 || workingDirectory.Length < 1)
throw new ArgumentException("Download-location and working-directory paths can not be empty!"); throw new ArgumentException("Download-location and working-directory paths can not be empty!");
@ -25,25 +26,53 @@ public class TrangaSettings
this.downloadLocation = downloadLocation; this.downloadLocation = downloadLocation;
this.libraryManagers = libraryManagers??new(); this.libraryManagers = libraryManagers??new();
this.notificationManagers = notificationManagers??new(); this.notificationManagers = notificationManagers??new();
this.logger = logger;
} }
public static TrangaSettings LoadSettings(string importFilePath, Logger? logger) public static TrangaSettings LoadSettings(string importFilePath, Logger? logger)
{ {
if (!File.Exists(importFilePath)) if (!File.Exists(importFilePath))
return new TrangaSettings(Path.Join(Directory.GetCurrentDirectory(), "Downloads"), return new TrangaSettings(Path.Join(Directory.GetCurrentDirectory(), "Downloads"),
Directory.GetCurrentDirectory(), new HashSet<LibraryManager>(), new HashSet<NotificationManager>()); Directory.GetCurrentDirectory(), new HashSet<LibraryManager>(), new HashSet<NotificationManager>(), logger);
string toRead = File.ReadAllText(importFilePath); string toRead = File.ReadAllText(importFilePath);
TrangaSettings settings = JsonConvert.DeserializeObject<TrangaSettings>(toRead, TrangaSettings settings = JsonConvert.DeserializeObject<TrangaSettings>(toRead,
new JsonSerializerSettings { Converters = { new NotificationManager.NotificationManagerJsonConverter(), new LibraryManager.LibraryManagerJsonConverter() } })!; new JsonSerializerSettings { Converters = { new NotificationManager.NotificationManagerJsonConverter(), new LibraryManager.LibraryManagerJsonConverter() } })!;
if (logger is not null) if (logger is not null)
{
foreach (LibraryManager lm in settings.libraryManagers) foreach (LibraryManager lm in settings.libraryManagers)
lm.AddLogger(logger); lm.AddLogger(logger);
foreach(NotificationManager nm in settings.notificationManagers)
nm.AddLogger(logger);
}
return settings; return settings;
} }
public void UpdateSettings(UpdateField field, Logger? logger = null, params string[] values) public void ExportSettings()
{
if (File.Exists(settingsFilePath))
{
bool inUse = true;
while (inUse)
{
try
{
using FileStream stream = new (settingsFilePath, FileMode.Open, FileAccess.Read, FileShare.None);
stream.Close();
inUse = false;
}
catch (IOException)
{
inUse = true;
Thread.Sleep(50);
}
}
}
File.WriteAllText(settingsFilePath, JsonConvert.SerializeObject(this));
}
public void UpdateSettings(UpdateField field, params string[] values)
{ {
switch (field) switch (field)
{ {
@ -56,19 +85,19 @@ public class TrangaSettings
if (values.Length != 2) if (values.Length != 2)
return; return;
libraryManagers.RemoveWhere(lm => lm.GetType() == typeof(Komga)); libraryManagers.RemoveWhere(lm => lm.GetType() == typeof(Komga));
libraryManagers.Add(new Komga(values[0], values[1], logger)); libraryManagers.Add(new Komga(values[0], values[1], this.logger));
break; break;
case UpdateField.Kavita: case UpdateField.Kavita:
if (values.Length != 3) if (values.Length != 3)
return; return;
libraryManagers.RemoveWhere(lm => lm.GetType() == typeof(Kavita)); libraryManagers.RemoveWhere(lm => lm.GetType() == typeof(Kavita));
libraryManagers.Add(new Kavita(values[0], values[1], values[2], logger)); libraryManagers.Add(new Kavita(values[0], values[1], values[2], this.logger));
break; break;
case UpdateField.Gotify: case UpdateField.Gotify:
if (values.Length != 2) if (values.Length != 2)
return; return;
notificationManagers.RemoveWhere(nm => nm.GetType() == typeof(Gotify)); notificationManagers.RemoveWhere(nm => nm.GetType() == typeof(Gotify));
Gotify newGotify = new(values[0], values[1], logger); Gotify newGotify = new(values[0], values[1], this.logger);
notificationManagers.Add(newGotify); notificationManagers.Add(newGotify);
newGotify.SendNotification("Success!", "Gotify was added to Tranga!"); newGotify.SendNotification("Success!", "Gotify was added to Tranga!");
break; break;
@ -76,11 +105,12 @@ public class TrangaSettings
if(values.Length != 1) if(values.Length != 1)
return; return;
notificationManagers.RemoveWhere(nm => nm.GetType() == typeof(LunaSea)); notificationManagers.RemoveWhere(nm => nm.GetType() == typeof(LunaSea));
LunaSea newLunaSea = new(values[0], logger); LunaSea newLunaSea = new(values[0], this.logger);
notificationManagers.Add(newLunaSea); notificationManagers.Add(newLunaSea);
newLunaSea.SendNotification("Success!", "LunaSea was added to Tranga!"); newLunaSea.SendNotification("Success!", "LunaSea was added to Tranga!");
break; break;
} }
ExportSettings();
} }
public enum UpdateField { DownloadLocation, Komga, Kavita, Gotify, LunaSea} public enum UpdateField { DownloadLocation, Komga, Kavita, Gotify, LunaSea}

View File

@ -1,5 +1,7 @@
using System.Net; using System.Net;
using Logging; using Tranga.Connectors;
using Tranga.NotificationManagers;
using Tranga.LibraryManagers;
namespace Tranga.TrangaTasks; namespace Tranga.TrangaTasks;
@ -20,16 +22,21 @@ public class DownloadChapterTask : TrangaTask
this.language = language; this.language = language;
} }
protected override HttpStatusCode ExecuteTask(TaskManager taskManager, Logger? logger, CancellationToken? cancellationToken = null) protected override HttpStatusCode ExecuteTask(TaskManager taskManager, CancellationToken? cancellationToken = null)
{ {
if (cancellationToken?.IsCancellationRequested ?? false) if (cancellationToken?.IsCancellationRequested ?? false)
return HttpStatusCode.RequestTimeout; return HttpStatusCode.RequestTimeout;
Connector connector = taskManager.GetConnector(this.connectorName); Connector connector = taskManager.GetConnector(this.connectorName);
connector.CopyCoverFromCacheToDownloadLocation(this.publication, taskManager.settings); connector.CopyCoverFromCacheToDownloadLocation(this.publication, taskManager.settings);
HttpStatusCode downloadSuccess = connector.DownloadChapter(this.publication, this.chapter, this, cancellationToken); HttpStatusCode downloadSuccess = connector.DownloadChapter(this.publication, this.chapter, this, cancellationToken);
if((int)downloadSuccess >= 200 && (int)downloadSuccess < 300 && parentTask is not null) if ((int)downloadSuccess >= 200 && (int)downloadSuccess < 300)
{
foreach(NotificationManager nm in taskManager.settings.notificationManagers) foreach(NotificationManager nm in taskManager.settings.notificationManagers)
nm.SendNotification("New Chapter downloaded", $"{this.publication.sortName} {this.chapter.chapterNumber} {this.chapter.name}"); nm.SendNotification("Chapter downloaded", $"{this.publication.sortName} {this.chapter.chapterNumber} {this.chapter.name}");
foreach (LibraryManager lm in taskManager.settings.libraryManagers)
lm.UpdateLibrary();
}
return downloadSuccess; return downloadSuccess;
} }
@ -47,6 +54,9 @@ public class DownloadChapterTask : TrangaTask
internal void IncrementProgress(double amount) internal void IncrementProgress(double amount)
{ {
this._dctProgress += amount; this._dctProgress += amount;
this.lastChange = DateTime.Now;
if(this.parentTask is not null)
this.parentTask.lastChange = DateTime.Now;
} }
public override string ToString() public override string ToString()

View File

@ -1,5 +1,5 @@
using System.Net; using System.Net;
using Logging; using Tranga.Connectors;
namespace Tranga.TrangaTasks; namespace Tranga.TrangaTasks;
@ -15,7 +15,7 @@ public class MonitorPublicationTask : TrangaTask
this.language = language; this.language = language;
} }
protected override HttpStatusCode ExecuteTask(TaskManager taskManager, Logger? logger, CancellationToken? cancellationToken = null) protected override HttpStatusCode ExecuteTask(TaskManager taskManager, CancellationToken? cancellationToken = null)
{ {
if (cancellationToken?.IsCancellationRequested ?? false) if (cancellationToken?.IsCancellationRequested ?? false)
return HttpStatusCode.RequestTimeout; return HttpStatusCode.RequestTimeout;
@ -23,11 +23,11 @@ public class MonitorPublicationTask : TrangaTask
//Check if Publication already has a Folder //Check if Publication already has a Folder
publication.CreatePublicationFolder(taskManager.settings.downloadLocation); publication.CreatePublicationFolder(taskManager.settings.downloadLocation);
List<Chapter> newChapters = taskManager.GetNewChaptersList(connector, publication, language); List<Chapter> newChapters = connector.GetNewChaptersList(publication, language, ref taskManager.collection);
connector.CopyCoverFromCacheToDownloadLocation(publication, taskManager.settings); connector.CopyCoverFromCacheToDownloadLocation(publication, taskManager.settings);
publication.SaveSeriesInfoJson(connector.downloadLocation); publication.SaveSeriesInfoJson(taskManager.settings.downloadLocation);
foreach (Chapter newChapter in newChapters) foreach (Chapter newChapter in newChapters)
{ {

View File

@ -3,10 +3,9 @@ using System.Text.Json.Serialization;
using Logging; using Logging;
using Newtonsoft.Json; using Newtonsoft.Json;
using Newtonsoft.Json.Linq; using Newtonsoft.Json.Linq;
using Tranga.TrangaTasks;
using JsonConverter = Newtonsoft.Json.JsonConverter; using JsonConverter = Newtonsoft.Json.JsonConverter;
namespace Tranga; namespace Tranga.TrangaTasks;
/// <summary> /// <summary>
/// Stores information on Task, when implementing new Tasks also update the serializer /// Stores information on Task, when implementing new Tasks also update the serializer
@ -22,14 +21,14 @@ public abstract class TrangaTask
public DateTime lastExecuted { get; set; } public DateTime lastExecuted { get; set; }
[Newtonsoft.Json.JsonIgnore] public ExecutionState state { get; set; } [Newtonsoft.Json.JsonIgnore] public ExecutionState state { get; set; }
public Task task { get; } public Task task { get; }
public string taskId { get; } public string taskId { get; init; }
[Newtonsoft.Json.JsonIgnore] public TrangaTask? parentTask { get; set; } [Newtonsoft.Json.JsonIgnore] public TrangaTask? parentTask { get; set; }
public string? parentTaskId { get; set; } public string? parentTaskId { get; set; }
[Newtonsoft.Json.JsonIgnore] protected HashSet<TrangaTask> childTasks { get; } [Newtonsoft.Json.JsonIgnore] internal HashSet<TrangaTask> childTasks { get; }
public double progress => GetProgress(); public double progress => GetProgress();
[Newtonsoft.Json.JsonIgnore]public DateTime executionStarted { get; private set; } [Newtonsoft.Json.JsonIgnore]public DateTime executionStarted { get; private set; }
[Newtonsoft.Json.JsonIgnore]public DateTime lastChange { get; private set; } [Newtonsoft.Json.JsonIgnore]public DateTime lastChange { get; internal set; }
[Newtonsoft.Json.JsonIgnore]public DateTime executionApproximatelyFinished => progress != 0 ? lastChange.Add(GetRemainingTime()) : DateTime.MaxValue; [Newtonsoft.Json.JsonIgnore]public DateTime executionApproximatelyFinished => lastChange.Add(GetRemainingTime());
public TimeSpan executionApproximatelyRemaining => executionApproximatelyFinished.Subtract(DateTime.Now); public TimeSpan executionApproximatelyRemaining => executionApproximatelyFinished.Subtract(DateTime.Now);
[Newtonsoft.Json.JsonIgnore]public DateTime nextExecution => lastExecuted.Add(reoccurrence); [Newtonsoft.Json.JsonIgnore]public DateTime nextExecution => lastExecuted.Add(reoccurrence);
@ -52,9 +51,8 @@ public abstract class TrangaTask
/// BL for concrete Tasks /// BL for concrete Tasks
/// </summary> /// </summary>
/// <param name="taskManager"></param> /// <param name="taskManager"></param>
/// <param name="logger"></param>
/// <param name="cancellationToken"></param> /// <param name="cancellationToken"></param>
protected abstract HttpStatusCode ExecuteTask(TaskManager taskManager, Logger? logger, CancellationToken? cancellationToken = null); protected abstract HttpStatusCode ExecuteTask(TaskManager taskManager, CancellationToken? cancellationToken = null);
public abstract TrangaTask Clone(); public abstract TrangaTask Clone();
@ -64,34 +62,33 @@ public abstract class TrangaTask
/// Execute the task /// Execute the task
/// </summary> /// </summary>
/// <param name="taskManager">Should be the parent taskManager</param> /// <param name="taskManager">Should be the parent taskManager</param>
/// <param name="logger"></param>
/// <param name="cancellationToken"></param> /// <param name="cancellationToken"></param>
public void Execute(TaskManager taskManager, Logger? logger, CancellationToken? cancellationToken = null) public void Execute(TaskManager taskManager, CancellationToken? cancellationToken = null)
{ {
logger?.WriteLine(this.GetType().ToString(), $"Executing Task {this}"); taskManager.settings.logger?.WriteLine(this.GetType().ToString(), $"Executing Task {this}");
this.state = ExecutionState.Running; this.state = ExecutionState.Running;
this.executionStarted = DateTime.Now; this.executionStarted = DateTime.Now;
this.lastChange = DateTime.Now; this.lastChange = DateTime.Now;
HttpStatusCode statusCode = ExecuteTask(taskManager, logger, cancellationToken); if(parentTask is not null && parentTask.childTasks.All(ct => ct.state is ExecutionState.Waiting or ExecutionState.Failed))
while(childTasks.Any(ct => ct.state is ExecutionState.Enqueued or ExecutionState.Running)) parentTask.executionStarted = DateTime.Now;
Thread.Sleep(1000);
HttpStatusCode statusCode = ExecuteTask(taskManager, cancellationToken);
if ((int)statusCode >= 200 && (int)statusCode < 300) if ((int)statusCode >= 200 && (int)statusCode < 300)
{ {
this.lastExecuted = DateTime.Now; this.lastExecuted = DateTime.Now;
if (this is DownloadChapterTask) this.state = ExecutionState.Success;
this.state = ExecutionState.Success;
else
this.state = ExecutionState.Waiting;
} }
else else
{ {
if (this is DownloadChapterTask && statusCode == HttpStatusCode.NotFound) this.state = ExecutionState.Failed;
this.state = ExecutionState.Success;
else
this.state = ExecutionState.Failed;
this.lastExecuted = DateTime.MaxValue; this.lastExecuted = DateTime.MaxValue;
} }
logger?.WriteLine(this.GetType().ToString(), $"Finished Executing Task {this}");
if (this is DownloadChapterTask)
taskManager.DeleteTask(this);
taskManager.settings.logger?.WriteLine(this.GetType().ToString(), $"Finished Executing Task {this}");
} }
public void AddChildTask(TrangaTask childTask) public void AddChildTask(TrangaTask childTask)
@ -106,10 +103,10 @@ public abstract class TrangaTask
private TimeSpan GetRemainingTime() private TimeSpan GetRemainingTime()
{ {
if(progress == 0 || lastChange == DateTime.MaxValue || executionStarted == DateTime.UnixEpoch) if(progress == 0 || state is ExecutionState.Enqueued or ExecutionState.Waiting or ExecutionState.Failed || lastChange == DateTime.MaxValue)
return TimeSpan.Zero; return DateTime.MaxValue.Subtract(lastChange).Subtract(TimeSpan.FromHours(1));
TimeSpan elapsed = lastChange.Subtract(executionStarted); TimeSpan elapsed = lastChange.Subtract(executionStarted);
return elapsed.Divide(progress).Subtract(elapsed); return elapsed.Divide(progress).Multiply(1 - progress);
} }
public enum Task : byte public enum Task : byte
@ -117,7 +114,6 @@ public abstract class TrangaTask
MonitorPublication = 2, MonitorPublication = 2,
UpdateLibraries = 3, UpdateLibraries = 3,
DownloadChapter = 4, DownloadChapter = 4,
DownloadNewChapters = 2 //legacy
} }
public override string ToString() public override string ToString()

View File

@ -3,6 +3,9 @@ using Logging;
namespace Tranga.TrangaTasks; namespace Tranga.TrangaTasks;
/// <summary>
/// LEGACY DEPRECATED
/// </summary>
public class UpdateLibrariesTask : TrangaTask public class UpdateLibrariesTask : TrangaTask
{ {
public UpdateLibrariesTask(TimeSpan reoccurrence) : base(Task.UpdateLibraries, reoccurrence) public UpdateLibrariesTask(TimeSpan reoccurrence) : base(Task.UpdateLibraries, reoccurrence)
@ -11,11 +14,7 @@ public class UpdateLibrariesTask : TrangaTask
protected override HttpStatusCode ExecuteTask(TaskManager taskManager, Logger? logger, CancellationToken? cancellationToken = null) protected override HttpStatusCode ExecuteTask(TaskManager taskManager, Logger? logger, CancellationToken? cancellationToken = null)
{ {
if (cancellationToken?.IsCancellationRequested ?? false) return HttpStatusCode.BadRequest;
return HttpStatusCode.RequestTimeout;
foreach(LibraryManager lm in taskManager.settings.libraryManagers)
lm.UpdateLibrary();
return HttpStatusCode.OK;
} }
public override TrangaTask Clone() public override TrangaTask Clone()

View File

@ -84,7 +84,7 @@ async function GetRunningTasks(){
} }
async function GetDownloadTasks(){ async function GetDownloadTasks(){
var uri = apiUri + "/Tasks?taskType=DownloadNewChapters"; var uri = apiUri + "/Tasks?taskType=MonitorPublication";
let json = await GetData(uri); let json = await GetData(uri);
return json; return json;
} }

View File

@ -142,7 +142,7 @@
<div> <div>
<span class="title">LunaSea</span> <span class="title">LunaSea</span>
<div>Configured: <span id="lunaseaConfigured">✅❌</span></div> <div>Configured: <span id="lunaseaConfigured">✅❌</span></div>
<label for="lunaseaWebhook"></label><input placeholder="Webhook-Url" id="lunaseaWebhook" type="text"> <label for="lunaseaWebhook"></label><input placeholder="device/:id or user/:id" id="lunaseaWebhook" type="text">
</div> </div>
<div> <div>
<label for="libraryUpdateTime" style="margin-right: 5px;">Update Time</label><input id="libraryUpdateTime" type="time" value="00:01:00" step="10"> <label for="libraryUpdateTime" style="margin-right: 5px;">Update Time</label><input id="libraryUpdateTime" type="time" value="00:01:00" step="10">

View File

@ -195,13 +195,13 @@ function DownloadChapterTaskClick(){
function DeleteTaskClick(){ function DeleteTaskClick(){
taskToDelete = tasks.filter(tTask => tTask.publication.internalId === toEditId)[0]; taskToDelete = tasks.filter(tTask => tTask.publication.internalId === toEditId)[0];
DeleteTask("DownloadNewChapters", taskToDelete.connectorName, toEditId); DeleteTask("MonitorPublication", taskToDelete.connectorName, toEditId);
HidePublicationPopup(); HidePublicationPopup();
} }
function StartTaskClick(){ function StartTaskClick(){
var toEditTask = tasks.filter(task => task.publication.internalId == toEditId)[0]; var toEditTask = tasks.filter(task => task.publication.internalId == toEditId)[0];
StartTask("DownloadNewChapters", toEditTask.connectorName, toEditId); StartTask("MonitorPublication", toEditTask.connectorName, toEditId);
HidePublicationPopup(); HidePublicationPopup();
} }

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.0 MiB

After

Width:  |  Height:  |  Size: 2.1 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 2.6 MiB

After

Width:  |  Height:  |  Size: 2.5 MiB

BIN
screenshots/progress.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 354 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 2.2 MiB

After

Width:  |  Height:  |  Size: 1.8 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.7 MiB

After

Width:  |  Height:  |  Size: 1.5 MiB