58 Commits
1.1.1 ... v1.3

Author SHA1 Message Date
7612411917 #33 Update Website 2023-06-03 16:25:04 +02:00
ed1402b5ec TrangaSettings Return libraryManageres on serialization 2023-06-03 16:24:54 +02:00
5adaee4821 redundant calls 2023-06-03 16:24:30 +02:00
2d82fe1489 libraryType in constructor 2023-06-03 16:24:14 +02:00
783fd8129e API: Kavita Auth #33 2023-06-03 15:40:26 +02:00
4f29eff48e Kavita authorization 2023-06-03 15:39:48 +02:00
e0e7abb62b #33 Added custom serializer for LibraryManager
Added Code for Kavita
2023-06-03 15:32:54 +02:00
ae63a216a5 unnecessary params 2023-06-03 15:27:35 +02:00
5d98295c59 #33 Preparation:
TrangaSettings now stores Hashset of LibraryManagers
2023-06-03 15:17:08 +02:00
0c580933f9 #33 Preparation:
Abstracted class Komga into LibraryManager
Fixed logger not attaching to LibraryManager
2023-06-03 15:02:15 +02:00
06f735aadd #32 API endpoint 2023-06-01 23:08:43 +02:00
439d69d8e0 Sanitization Manganto 2023-06-01 22:59:51 +02:00
933df58712 Moved publicationFolder creation to Publication with Permissions 2023-06-01 22:59:04 +02:00
165bbc412e Adjusted Manganato ratelimit 2023-06-01 22:49:20 +02:00
6158fa072b File permissions 2023-06-01 22:32:11 +02:00
0d3799e00d Fix Bug when strings where shorter than 25 characters on logger.writeline
Fixed CLI output
2023-06-01 22:27:37 +02:00
e977bed5a5 #32 formatting length 2023-06-01 22:14:00 +02:00
cacd5fede2 removed unnecessary todo 2023-06-01 22:06:10 +02:00
1bca99cb6a #32 Added progress tracking to task (internal and log use for now) 2023-06-01 22:05:48 +02:00
15fc367263 logging 2023-06-01 21:16:57 +02:00
8bb6fb902b File Permissions 2023-06-01 18:28:58 +02:00
a57903cd5a readme update 2023-06-01 16:04:21 +02:00
1cd819b21d update docker-compose 2023-06-01 15:40:14 +02:00
27afedc1b4 year in series.json 2023-06-01 15:25:26 +02:00
fac0a3f7eb resolves #2 2023-06-01 15:08:32 +02:00
03ca480fe8 remove empty lines at start of description 2023-06-01 14:59:09 +02:00
c2915468a5 status .tolower 2023-06-01 14:58:58 +02:00
8805c53cb8 wrong url for manga info page 2023-06-01 13:37:06 +02:00
adbbe3f6cc logs 2023-06-01 13:27:58 +02:00
14b694d3be Description value duplicate #2 2023-06-01 13:25:58 +02:00
72ce75c6e0 #2 Multiple alt titles 2023-06-01 13:18:26 +02:00
8381951168 #2 First Attempt 2023-06-01 13:13:53 +02:00
b24032d124 remove env var 2023-06-01 11:29:47 +02:00
8bc23f7c69 Instead of relying on concreate tasks to do chores, create method in abstract class that calls the BL in concrete class and does chores before and after execution 2023-06-01 10:35:23 +02:00
48b7371a18 Issue missing parameter 2023-05-31 22:34:13 +02:00
61ecefb615 Logging and Chores in abstract class 2023-05-31 22:32:37 +02:00
8ff65bf400 compatibility with older tasks.json 2023-05-31 22:26:53 +02:00
932057cca0 update execution time 2023-05-31 22:20:33 +02:00
67d06cd887 resolves #23 website filter 2023-05-31 22:16:14 +02:00
cbb012a659 alt and label 2023-05-31 22:16:01 +02:00
e4f33bcca9 Set ExecutionState to waiting after finished 2023-05-31 21:52:50 +02:00
fbba7c45b9 annotation 2023-05-31 21:44:16 +02:00
d9b6062767 Custom JSON Deserializer for concrete classes of abstract class TrangaTask 2023-05-31 21:43:07 +02:00
d477cd1ccd corrected check for tasktype on due 2023-05-31 21:42:23 +02:00
f892db7dda Switch Execution State to Running when executing 2023-05-31 21:40:00 +02:00
16c1b5c506 Create imageCacheFolder 2023-05-31 21:39:18 +02:00
d5ecc1c37d Spelling, redundant calls 2023-05-31 21:18:41 +02:00
1b9ebd096b Removed TaskExecutor
TrangaTask is now abstract
TrangaTask implements Execute Method, that is now called instead of TaskExecutor
Created inheriting classes of TrangaTask: UpdateKomgaLibraryTask, DownloadNewChaptersTask
2023-05-31 21:15:32 +02:00
8619630269 Renamed ExportData to ExportDataAndSettings 2023-05-31 21:14:11 +02:00
2bc92556e3 Method description and Name CopyCoverFromCacheToDownloadLocation 2023-05-31 20:39:57 +02:00
f1ab823e7f Method description 2023-05-31 20:39:23 +02:00
8261d02cc7 Renamed methods, made some methods non-static in Connector, some more logging 2023-05-31 20:29:30 +02:00
8d3b8be95c settings placeholders for komga if configured 2023-05-31 17:55:09 +02:00
60519910de cursors 2023-05-31 17:54:54 +02:00
0940afe41f publication viewer tags and styling 2023-05-31 17:54:09 +02:00
3dc376c19f favicon 2023-05-31 17:52:47 +02:00
3e56ef842b Update docker-compose.yaml with version 2023-05-30 19:37:09 +02:00
5a44e3b8b9 #25 only replace settings if parameter actually contains value. 2023-05-30 19:32:22 +02:00
25 changed files with 975 additions and 525 deletions

View File

@ -54,6 +54,7 @@
Tranga can download Chapters and Metadata from Scanlation sites such as
- [MangaDex.org](https://mangadex.org/)
- [Manganato.com](https://manganato.com/)
and automatically start updates in [Komga](https://komga.org/) to import them.
@ -114,7 +115,7 @@ Wherever you are mounting `/usr/share/Tranga-API` you also need to mount that sa
- [x] Web-UI #1
- [ ] More Connectors
- [ ] Manganato #2
- [x] Manganato #2
- [ ] ?
See the [open issues](https://git.bernloehr.eu/glax/Tranga/issues) for a full list of proposed features (and known issues).

View File

@ -15,9 +15,9 @@ logger.WriteLine("Tranga", "Loading settings.");
TrangaSettings settings;
if (File.Exists(settingsFilePath))
settings = TrangaSettings.LoadSettings(settingsFilePath);
settings = TrangaSettings.LoadSettings(settingsFilePath, logger);
else
settings = new TrangaSettings(downloadFolderPath, applicationFolderPath, null);
settings = new TrangaSettings(downloadFolderPath, applicationFolderPath, new HashSet<LibraryManager>());
Directory.CreateDirectory(settings.workingDirectory);
Directory.CreateDirectory(settings.downloadLocation);
@ -51,8 +51,6 @@ builder.Services.AddCors(options =>
var app = builder.Build();
app.UseSwagger();
app.UseSwaggerUI();
app.UseSwagger();
app.UseSwaggerUI();
app.UseCors(corsHeader);
@ -105,6 +103,31 @@ app.MapGet("/Tasks/Get", (string taskType, string? connectorName, string? search
}
});
app.MapGet("/Tasks/GetTaskProgress", (string taskType, string? connectorName, string? publicationId) =>
{
try
{
TrangaTask.Task pTask = Enum.Parse<TrangaTask.Task>(taskType);
TrangaTask? task = null;
if (connectorName is null || publicationId is null)
task = taskManager.GetAllTasks().FirstOrDefault(tTask =>
tTask.task == pTask);
else
task = taskManager.GetAllTasks().FirstOrDefault(tTask =>
tTask.task == pTask && tTask.publication?.internalId == publicationId &&
tTask.connectorName == connectorName);
if (task is null)
return -1f;
return task.progress;
}
catch (ArgumentException)
{
return -1f;
}
});
app.MapPost("/Tasks/Start", (string taskType, string? connectorName, string? publicationId) =>
{
try
@ -186,6 +209,8 @@ app.MapDelete("/Queue/Dequeue", (string taskType, string? connectorName, string?
app.MapGet("/Settings/Get", () => taskManager.settings);
app.MapPost("/Settings/Update", (string? downloadLocation, string? komgaUrl, string? komgaAuth) => taskManager.UpdateSettings(downloadLocation, komgaUrl, komgaAuth) );
app.MapPost("/Settings/Update",
(string? downloadLocation, string? komgaUrl, string? komgaAuth, string? kavitaUrl, string? kavitaApiKey) =>
taskManager.UpdateSettings(downloadLocation, komgaUrl, komgaAuth, kavitaUrl, kavitaApiKey));
app.Run();

View File

@ -12,26 +12,17 @@
"commandName": "Project",
"dotnetRunMessages": true,
"launchBrowser": true,
"applicationUrl": "http://localhost:5177",
"environmentVariables": {
"ASPNETCORE_ENVIRONMENT": "Development"
}
"applicationUrl": "http://localhost:5177"
},
"https": {
"commandName": "Project",
"dotnetRunMessages": true,
"launchBrowser": true,
"applicationUrl": "https://localhost:7036;http://localhost:5177",
"environmentVariables": {
"ASPNETCORE_ENVIRONMENT": "Development"
}
"applicationUrl": "https://localhost:7036;http://localhost:5177"
},
"IIS Express": {
"commandName": "IISExpress",
"launchBrowser": true,
"environmentVariables": {
"ASPNETCORE_ENVIRONMENT": "Development"
}
"launchBrowser": true
}
}
}

View File

@ -1,6 +1,7 @@
using System.Globalization;
using Logging;
using Tranga;
using Tranga.LibraryManagers;
namespace Tranga_CLI;
@ -25,14 +26,10 @@ public static class Tranga_Cli
Console.WriteLine($"Logfile-Path: {logFilePath}");
Console.WriteLine($"Settings-File-Path: {settingsFilePath}");
Logger logger = new(new[] { Logger.LoggerType.FileLogger }, null, null, logFilePath);
Logger logger = new(new[] { Logger.LoggerType.FileLogger }, null, Console.Out.Encoding, logFilePath);
logger.WriteLine("Tranga_CLI", "Loading Taskmanager.");
TrangaSettings settings;
if (File.Exists(settingsFilePath))
settings = TrangaSettings.LoadSettings(settingsFilePath);
else
settings = new TrangaSettings(Directory.GetCurrentDirectory(), applicationFolderPath, null);
TrangaSettings settings = File.Exists(settingsFilePath) ? TrangaSettings.LoadSettings(settingsFilePath, logger) : new TrangaSettings(Directory.GetCurrentDirectory(), applicationFolderPath, new HashSet<LibraryManager>());
logger.WriteLine("Tranga_CLI", "User Input");
@ -43,11 +40,11 @@ public static class Tranga_Cli
if (tmpPath.Length > 0)
settings.downloadLocation = tmpPath;
Console.WriteLine($"Komga BaseURL [{settings.komga?.baseUrl}]:");
string? tmpUrl = Console.ReadLine();
while (tmpUrl is null)
tmpUrl = Console.ReadLine();
if (tmpUrl.Length > 0)
Console.WriteLine($"Komga BaseURL [{settings.libraryManagers.FirstOrDefault(lm => lm.GetType() == typeof(Komga))?.baseUrl}]:");
string? tmpUrlKomga = Console.ReadLine();
while (tmpUrlKomga is null)
tmpUrlKomga = Console.ReadLine();
if (tmpUrlKomga.Length > 0)
{
Console.WriteLine("Username:");
string? tmpUser = Console.ReadLine();
@ -74,7 +71,23 @@ public static class Tranga_Cli
}
} while (key != ConsoleKey.Enter);
settings.komga = new Komga(tmpUrl, tmpUser, tmpPass, logger);
settings.libraryManagers.RemoveWhere(lm => lm.GetType() == typeof(Komga));
settings.libraryManagers.Add(new Komga(tmpUrlKomga, tmpUser, tmpPass, logger));
}
Console.WriteLine($"Kavita BaseURL [{settings.libraryManagers.FirstOrDefault(lm => lm.GetType() == typeof(Kavita))?.baseUrl}]:");
string? tmpUrlKavita = Console.ReadLine();
while (tmpUrlKavita is null)
tmpUrlKavita = Console.ReadLine();
if (tmpUrlKavita.Length > 0)
{
Console.WriteLine("Username:");
string? tmpApiKey = Console.ReadLine();
while (tmpApiKey is null || tmpApiKey.Length < 1)
tmpApiKey = Console.ReadLine();
settings.libraryManagers.RemoveWhere(lm => lm.GetType() == typeof(Kavita));
settings.libraryManagers.Add(new Kavita(tmpUrlKavita, tmpApiKey, logger));
}
logger.WriteLine("Tranga_CLI", "Loaded.");
@ -85,7 +98,7 @@ public static class Tranga_Cli
{
TaskManager taskManager = new (settings, logger);
ConsoleKey selection = ConsoleKey.EraseEndOfFile;
PrintMenu(taskManager, taskManager.settings.downloadLocation, logger);
PrintMenu(taskManager, taskManager.settings.downloadLocation);
while (selection != ConsoleKey.Q)
{
int taskCount = taskManager.GetAllTasks().Length;
@ -106,7 +119,7 @@ public static class Tranga_Cli
Console.ReadKey();
break;
case ConsoleKey.C:
CreateTask(taskManager, taskManager.settings, logger);
CreateTask(taskManager, logger);
Console.WriteLine("Press any key.");
Console.ReadKey();
break;
@ -159,7 +172,7 @@ public static class Tranga_Cli
Console.ReadKey();
break;
}
PrintMenu(taskManager, taskManager.settings.downloadLocation, logger);
PrintMenu(taskManager, taskManager.settings.downloadLocation);
}
Thread.Sleep(200);
}
@ -179,7 +192,7 @@ public static class Tranga_Cli
taskManager.Shutdown(false);
}
private static void PrintMenu(TaskManager taskManager, string folderPath, Logger logger)
private static void PrintMenu(TaskManager taskManager, string folderPath)
{
int taskCount = taskManager.GetAllTasks().Length;
int taskRunningCount = taskManager.GetAllTasks().Count(task => task.state == TrangaTask.ExecutionState.Running);
@ -206,13 +219,13 @@ public static class Tranga_Cli
int tIndex = 0;
Console.WriteLine($"Tasks (Running/Queue/Total): {taskRunningCount}/{taskEnqueuedCount}/{taskCount}");
string header =
$"{"",-5}{"Task",-20} | {"Last Executed",-20} | {"Reoccurrence",-12} | {"State",-10} | {"Connector",-15} | Publication/Manga";
$"{"",-5}{"Task",-20} | {"Last Executed",-20} | {"Reoccurrence",-12} | {"State",-10} | {"Connector",-15} | {"Progress",-9} | Publication/Manga";
Console.WriteLine(header);
Console.WriteLine(new string('-', header.Length));
foreach (TrangaTask trangaTask in tasks)
{
string[] taskSplit = trangaTask.ToString().Split(", ");
Console.WriteLine($"{tIndex++:000}: {taskSplit[0],-20} | {taskSplit[1],-20} | {taskSplit[2],-12} | {taskSplit[3],-10} | {(taskSplit.Length > 4 ? taskSplit[4] : ""),-15} | {(taskSplit.Length > 5 ? taskSplit[5] : "")}");
Console.WriteLine($"{tIndex++:000}: {taskSplit[0],-20} | {taskSplit[1],-20} | {taskSplit[2],-12} | {taskSplit[3],-10} | {(taskSplit.Length > 4 ? taskSplit[4] : ""),-15} | {(taskSplit.Length > 5 ? taskSplit[5] : ""),-9} |{(taskSplit.Length > 6 ? taskSplit[6] : "")}");
}
}
@ -264,17 +277,17 @@ public static class Tranga_Cli
Console.Clear();
logger.WriteLine("Tranga_CLI", "Menu: Add Manga Download to queue");
Connector? connector = SelectConnector(taskManager.settings.downloadLocation, taskManager.GetAvailableConnectors().Values.ToArray(), logger);
Connector? connector = SelectConnector(taskManager.GetAvailableConnectors().Values.ToArray(), logger);
if (connector is null)
return;
Publication? publication = SelectPublication(taskManager, connector!, logger);
Publication? publication = SelectPublication(taskManager, connector, logger);
if (publication is null)
return;
TimeSpan reoccurrence = SelectReoccurrence(logger);
logger.WriteLine("Tranga_CLI", "Sending Task to TaskManager");
TrangaTask newTask = taskManager.AddTask(TrangaTask.Task.DownloadNewChapters, connector?.name, publication, reoccurrence, "en");
TrangaTask newTask = taskManager.AddTask(TrangaTask.Task.DownloadNewChapters, connector.name, publication, reoccurrence, "en");
Console.WriteLine(newTask);
}
@ -327,24 +340,24 @@ public static class Tranga_Cli
}
}
private static void CreateTask(TaskManager taskManager, TrangaSettings settings, Logger logger)
private static void CreateTask(TaskManager taskManager, Logger logger)
{
logger.WriteLine("Tranga_CLI", "Menu: Creating Task");
TrangaTask.Task? tmpTask = SelectTaskType(logger);
if (tmpTask is null)
return;
TrangaTask.Task task = (TrangaTask.Task)tmpTask!;
TrangaTask.Task task = (TrangaTask.Task)tmpTask;
Connector? connector = null;
if (task != TrangaTask.Task.UpdateKomgaLibrary)
if (task != TrangaTask.Task.UpdateLibraries)
{
connector = SelectConnector(settings.downloadLocation, taskManager.GetAvailableConnectors().Values.ToArray(), logger);
connector = SelectConnector(taskManager.GetAvailableConnectors().Values.ToArray(), logger);
if (connector is null)
return;
}
Publication? publication = null;
if (task != TrangaTask.Task.UpdatePublications && task != TrangaTask.Task.UpdateKomgaLibrary)
if (task != TrangaTask.Task.UpdateLibraries)
{
publication = SelectPublication(taskManager, connector!, logger);
if (publication is null)
@ -431,7 +444,7 @@ public static class Tranga_Cli
return TimeSpan.Parse(Console.ReadLine()!, new CultureInfo("en-US"));
}
private static Connector? SelectConnector(string folderPath, Connector[] connectors, Logger logger)
private static Connector? SelectConnector(Connector[] connectors, Logger logger)
{
logger.WriteLine("Tranga_CLI", "Menu: Select Connector");
Console.Clear();

View File

@ -1,4 +1,5 @@
<wpf:ResourceDictionary xml:space="preserve" xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" xmlns:s="clr-namespace:System;assembly=mscorlib" xmlns:ss="urn:shemas-jetbrains-com:settings-storage-xaml" xmlns:wpf="http://schemas.microsoft.com/winfx/2006/xaml/presentation">
<s:Boolean x:Key="/Default/UserDictionary/Words/=Komga/@EntryIndexedValue">True</s:Boolean>
<s:Boolean x:Key="/Default/UserDictionary/Words/=Manganato/@EntryIndexedValue">True</s:Boolean>
<s:Boolean x:Key="/Default/UserDictionary/Words/=Taskmanager/@EntryIndexedValue">True</s:Boolean>
<s:Boolean x:Key="/Default/UserDictionary/Words/=Tranga/@EntryIndexedValue">True</s:Boolean></wpf:ResourceDictionary>

View File

@ -1,7 +1,9 @@
using System.IO.Compression;
using System.Net;
using System.Runtime.InteropServices;
using System.Xml.Linq;
using Logging;
using static System.IO.UnixFileMode;
namespace Tranga;
@ -14,9 +16,9 @@ public abstract class Connector
internal string downloadLocation { get; } //Location of local files
protected DownloadClient downloadClient { get; init; }
protected Logger? logger;
protected readonly Logger? logger;
protected string imageCachePath;
protected readonly string imageCachePath;
protected Connector(string downloadLocation, string imageCachePath, Logger? logger)
{
@ -27,6 +29,8 @@ public abstract class Connector
//RequestTypes for RateLimits
}, logger);
this.imageCachePath = imageCachePath;
if (!Directory.Exists(imageCachePath))
Directory.CreateDirectory(this.imageCachePath);
}
public abstract string name { get; } //Name of the Connector (e.g. Website)
@ -50,34 +54,36 @@ public abstract class Connector
/// <summary>
/// Retrieves the Chapter (+Images) from the website.
/// Should later call DownloadChapterImages to retrieve the individual Images of the Chapter.
/// Should later call DownloadChapterImages to retrieve the individual Images of the Chapter and create .cbz archive.
/// </summary>
/// <param name="publication">Publication that contains Chapter</param>
/// <param name="chapter">Chapter with Images to retrieve</param>
public abstract void DownloadChapter(Publication publication, Chapter chapter);
/// <param name="parentTask">Will be used for progress-tracking</param>
public abstract void DownloadChapter(Publication publication, Chapter chapter, TrangaTask parentTask);
/// <summary>
/// Retrieves the Cover from the Website
/// Copies the already downloaded cover from cache to downloadLocation
/// </summary>
/// <param name="publication">Publication to retrieve Cover for</param>
/// <param name="settings">TrangaSettings</param>
public abstract void CloneCoverFromCache(Publication publication, TrangaSettings settings);
/// <summary>
/// Saves the series-info to series.json in the Publication Folder
/// </summary>
/// <param name="publication">Publication to save series.json for</param>
public void SaveSeriesInfo(Publication publication)
public void CopyCoverFromCacheToDownloadLocation(Publication publication, TrangaSettings settings)
{
logger?.WriteLine(this.GetType().ToString(), $"Saving series.json for {publication.sortName}");
//Check if Publication already has a Folder and a series.json
string publicationFolder = Path.Join(downloadLocation, publication.folderName);
if(!Directory.Exists(publicationFolder))
Directory.CreateDirectory(publicationFolder);
logger?.WriteLine(this.GetType().ToString(), $"Cloning cover {publication.sortName} {publication.internalId}");
//Check if Publication already has a Folder and cover
string publicationFolder = publication.CreatePublicationFolder(downloadLocation);
DirectoryInfo dirInfo = new (publicationFolder);
if (dirInfo.EnumerateFiles().Any(info => info.Name.Contains("cover.")))
{
logger?.WriteLine(this.GetType().ToString(), $"Cover exists {publication.sortName}");
return;
}
string seriesInfoPath = Path.Join(publicationFolder, "series.json");
if(!File.Exists(seriesInfoPath))
File.WriteAllText(seriesInfoPath,publication.GetSeriesInfoJson());
string fileInCache = Path.Join(settings.coverImageCache, publication.coverFileNameInCache);
string newFilePath = Path.Join(publicationFolder, $"cover.{Path.GetFileName(fileInCache).Split('.')[^1]}" );
logger?.WriteLine(this.GetType().ToString(), $"Cloning cover {fileInCache} -> {newFilePath}");
File.Copy(fileInCache, newFilePath, true);
if(RuntimeInformation.IsOSPlatform(OSPlatform.Linux))
File.SetUnixFileMode(newFilePath, GroupRead | GroupWrite | OtherRead | OtherWrite | UserRead | UserWrite);
}
/// <summary>
@ -85,7 +91,7 @@ public abstract class Connector
/// See ComicInfo.xml
/// </summary>
/// <returns>XML-string</returns>
protected static string CreateComicInfo(Publication publication, Chapter chapter, Logger? logger)
protected static string GetComicInfoXmlString(Publication publication, Chapter chapter, Logger? logger)
{
logger?.WriteLine("Connector", $"Creating ComicInfo.Xml for {publication.sortName} {publication.internalId} {chapter.volumeNumber}-{chapter.chapterNumber}");
XElement comicInfo = new XElement("ComicInfo",
@ -94,7 +100,7 @@ public abstract class Connector
new XElement("Title", chapter.name),
new XElement("Writer", publication.author),
new XElement("Volume", chapter.volumeNumber),
new XElement("Number", chapter.chapterNumber) //TODO check if this is correct at some point
new XElement("Number", chapter.chapterNumber)
);
return comicInfo.ToString();
}
@ -103,16 +109,16 @@ public abstract class Connector
/// Checks if a chapter-archive is already present
/// </summary>
/// <returns>true if chapter is present</returns>
public bool ChapterIsDownloaded(Publication publication, Chapter chapter)
public bool CheckChapterIsDownloaded(Publication publication, Chapter chapter)
{
return File.Exists(CreateFullFilepath(publication, chapter));
return File.Exists(GetArchiveFilePath(publication, chapter));
}
/// <summary>
/// Creates full file path of chapter-archive
/// </summary>
/// <returns>Filepath</returns>
protected string CreateFullFilepath(Publication publication, Chapter chapter)
protected string GetArchiveFilePath(Publication publication, Chapter chapter)
{
return Path.Join(downloadLocation, publication.folderName, $"{chapter.fileName}.cbz");
}
@ -122,14 +128,18 @@ public abstract class Connector
/// </summary>
/// <param name="imageUrl"></param>
/// <param name="fullPath"></param>
/// <param name="downloadClient">DownloadClient of the connector</param>
/// <param name="requestType">Requesttype for ratelimit</param>
protected static void DownloadImage(string imageUrl, string fullPath, DownloadClient downloadClient, byte requestType)
/// <param name="requestType">RequestType for Rate-Limit</param>
/// <param name="referrer">referrer used in html request header</param>
private void DownloadImage(string imageUrl, string fullPath, byte requestType, string? referrer = null)
{
DownloadClient.RequestResult requestResult = downloadClient.MakeRequest(imageUrl, requestType, referrer);
if (requestResult.result != Stream.Null)
{
DownloadClient.RequestResult requestResult = downloadClient.MakeRequest(imageUrl, requestType);
byte[] buffer = new byte[requestResult.result.Length];
requestResult.result.ReadExactly(buffer, 0, buffer.Length);
File.WriteAllBytes(fullPath, buffer);
}else
logger?.WriteLine(this.GetType().ToString(), "No Stream-Content in result.");
}
/// <summary>
@ -137,11 +147,11 @@ public abstract class Connector
/// </summary>
/// <param name="imageUrls">List of URLs to download Images from</param>
/// <param name="saveArchiveFilePath">Full path to save archive to (without file ending .cbz)</param>
/// <param name="downloadClient">DownloadClient of the connector</param>
/// <param name="logger"></param>
/// <param name="parentTask">Used for progress tracking</param>
/// <param name="comicInfoPath">Path of the generate Chapter ComicInfo.xml, if it was generated</param>
/// <param name="requestType">RequestType for RateLimits</param>
protected static void DownloadChapterImages(string[] imageUrls, string saveArchiveFilePath, DownloadClient downloadClient, byte requestType, Logger? logger, string? comicInfoPath = null)
/// <param name="referrer">Used in http request header</param>
protected void DownloadChapterImages(string[] imageUrls, string saveArchiveFilePath, byte requestType, TrangaTask parentTask, string? comicInfoPath = null, string? referrer = null)
{
logger?.WriteLine("Connector", $"Downloading Images for {saveArchiveFilePath}");
//Check if Publication Directory already exists
@ -161,8 +171,9 @@ public abstract class Connector
{
string[] split = imageUrl.Split('.');
string extension = split[^1];
logger?.WriteLine("Connector", $"Downloading Image {chapter + 1}/{imageUrls.Length}");
DownloadImage(imageUrl, Path.Join(tempFolder, $"{chapter++}.{extension}"), downloadClient, requestType);
logger?.WriteLine("Connector", $"Downloading Image {chapter + 1:000}/{imageUrls.Length:000} {(parentTask.publication?.sortName)![..(int)(parentTask.publication?.sortName.Length > 25 ? 25 : parentTask.publication?.sortName.Length)!],-25} {(parentTask.publication?.internalId)![..(int)(parentTask.publication?.internalId.Length > 25 ? 25 : parentTask.publication?.internalId.Length)!],-25} Total Task Progress: {parentTask.progress:00.0}%");
DownloadImage(imageUrl, Path.Join(tempFolder, $"{chapter++}.{extension}"), requestType, referrer);
parentTask.tasksFinished++;
}
if(comicInfoPath is not null)
@ -171,9 +182,28 @@ public abstract class Connector
logger?.WriteLine("Connector", $"Creating archive {saveArchiveFilePath}");
//ZIP-it and ship-it
ZipFile.CreateFromDirectory(tempFolder, saveArchiveFilePath);
if(RuntimeInformation.IsOSPlatform(OSPlatform.Linux))
File.SetUnixFileMode(saveArchiveFilePath, GroupRead | GroupWrite | OtherRead | OtherWrite | UserRead | UserWrite);
Directory.Delete(tempFolder, true); //Cleanup
}
protected string SaveCoverImageToCache(string url, byte requestType)
{
string[] split = url.Split('/');
string filename = split[^1];
string saveImagePath = Path.Join(imageCachePath, filename);
if (File.Exists(saveImagePath))
return filename;
DownloadClient.RequestResult coverResult = downloadClient.MakeRequest(url, requestType);
using MemoryStream ms = new();
coverResult.result.CopyTo(ms);
File.WriteAllBytes(saveImagePath, ms.ToArray());
logger?.WriteLine(this.GetType().ToString(), $"Saving image to {saveImagePath}");
return filename;
}
protected class DownloadClient
{
private static readonly HttpClient Client = new();
@ -185,8 +215,8 @@ public abstract class Connector
/// <summary>
/// Creates a httpClient
/// </summary>
/// <param name="delay">minimum delay between requests (to avoid spam)</param>
/// <param name="rateLimitRequestsPerMinute">Rate limits for requests. byte is RequestType, int maximum requests per minute for RequestType</param>
/// <param name="logger"></param>
public DownloadClient(Dictionary<byte, int> rateLimitRequestsPerMinute, Logger? logger)
{
this.logger = logger;
@ -201,8 +231,9 @@ public abstract class Connector
/// </summary>
/// <param name="url"></param>
/// <param name="requestType">For RateLimits: Same Endpoints use same type</param>
/// <param name="referrer">Used in http request header</param>
/// <returns>RequestResult with StatusCode and Stream of received data</returns>
public RequestResult MakeRequest(string url, byte requestType)
public RequestResult MakeRequest(string url, byte requestType, string? referrer = null)
{
if (_rateLimit.TryGetValue(requestType, out TimeSpan value))
_lastExecutedRateLimit.TryAdd(requestType, DateTime.Now.Subtract(value));
@ -224,13 +255,15 @@ public abstract class Connector
try
{
HttpRequestMessage requestMessage = new(HttpMethod.Get, url);
if(referrer is not null)
requestMessage.Headers.Referrer = new Uri(referrer);
_lastExecutedRateLimit[requestType] = DateTime.Now;
response = Client.Send(requestMessage);
}
catch (HttpRequestException e)
{
logger?.WriteLine(this.GetType().ToString(), e.Message);
logger?.WriteLine(this.GetType().ToString(), $"Waiting {_rateLimit[requestType] * 2}");
logger?.WriteLine(this.GetType().ToString(), $"Waiting {_rateLimit[requestType] * 2}... Retrying.");
Thread.Sleep(_rateLimit[requestType] * 2);
}
}

View File

@ -38,6 +38,7 @@ public class MangaDex : Connector
int offset = 0; //"Page"
int total = int.MaxValue; //How many total results are there, is updated on first request
HashSet<Publication> publications = new();
int loadedPublicationData = 0;
while (offset < total) //As long as we haven't requested all "Pages"
{
//Request next Page
@ -55,10 +56,10 @@ public class MangaDex : Connector
total = result["total"]!.GetValue<int>(); //Update the total number of Publications
JsonArray mangaInResult = result["data"]!.AsArray(); //Manga-data-Array
logger?.WriteLine(this.GetType().ToString(), $"Getting publication data.");
//Loop each Manga and extract information from JSON
foreach (JsonNode? mangeNode in mangaInResult)
{
logger?.WriteLine(this.GetType().ToString(), $"Getting publication data. {++loadedPublicationData}/{total}");
JsonObject manga = (JsonObject)mangeNode!;
JsonObject attributes = manga["attributes"]!.AsObject();
@ -101,7 +102,7 @@ public class MangaDex : Connector
string? coverUrl = GetCoverUrl(publicationId, posterId);
string? coverCacheName = null;
if (coverUrl is not null)
coverCacheName = SaveImage(coverUrl);
coverCacheName = SaveCoverImageToCache(coverUrl, (byte)RequestType.AtHomeServer);
string? author = GetAuthor(authorId);
@ -203,7 +204,7 @@ public class MangaDex : Connector
return chapters.OrderBy(chapter => Convert.ToSingle(chapter.chapterNumber, chapterNumberFormatInfo)).ToArray();
}
public override void DownloadChapter(Publication publication, Chapter chapter)
public override void DownloadChapter(Publication publication, Chapter chapter, TrangaTask parentTask)
{
logger?.WriteLine(this.GetType().ToString(), $"Downloading Chapter-Info {publication.sortName} {publication.internalId} {chapter.volumeNumber}-{chapter.chapterNumber}");
//Request URLs for Chapter-Images
@ -224,10 +225,10 @@ public class MangaDex : Connector
imageUrls.Add($"{baseUrl}/data/{hash}/{image!.GetValue<string>()}");
string comicInfoPath = Path.GetTempFileName();
File.WriteAllText(comicInfoPath, CreateComicInfo(publication, chapter, logger));
File.WriteAllText(comicInfoPath, GetComicInfoXmlString(publication, chapter, logger));
//Download Chapter-Images
DownloadChapterImages(imageUrls.ToArray(), CreateFullFilepath(publication, chapter), downloadClient, (byte)RequestType.AtHomeServer, logger, comicInfoPath);
DownloadChapterImages(imageUrls.ToArray(), GetArchiveFilePath(publication, chapter), (byte)RequestType.AtHomeServer, parentTask, comicInfoPath);
}
private string? GetCoverUrl(string publicationId, string? posterId)
@ -272,42 +273,4 @@ public class MangaDex : Connector
logger?.WriteLine(this.GetType().ToString(), $"Got author {authorId} -> {author}");
return author;
}
public override void CloneCoverFromCache(Publication publication, TrangaSettings settings)
{
logger?.WriteLine(this.GetType().ToString(), $"Cloning cover {publication.sortName}");
//Check if Publication already has a Folder and cover
string publicationFolder = Path.Join(downloadLocation, publication.folderName);
if(!Directory.Exists(publicationFolder))
Directory.CreateDirectory(publicationFolder);
DirectoryInfo dirInfo = new (publicationFolder);
if (dirInfo.EnumerateFiles().Any(info => info.Name.Contains("cover.")))
{
logger?.WriteLine(this.GetType().ToString(), $"Cover exists {publication.sortName}");
return;
}
string fileInCache = Path.Join(settings.coverImageCache, publication.coverFileNameInCache);
string newFilePath = Path.Join(publicationFolder, $"cover.{Path.GetFileName(fileInCache).Split('.')[^1]}" );
logger?.WriteLine(this.GetType().ToString(), $"Cloning cover {fileInCache} -> {newFilePath}");
File.Copy(fileInCache, newFilePath, true);
}
private string SaveImage(string url)
{
string[] split = url.Split('/');
string filename = split[^1];
string saveImagePath = Path.Join(imageCachePath, filename);
if (File.Exists(saveImagePath))
return filename;
DownloadClient.RequestResult coverResult = downloadClient.MakeRequest(url, (byte)RequestType.AtHomeServer);
using MemoryStream ms = new();
coverResult.result.CopyTo(ms);
File.WriteAllBytes(saveImagePath, ms.ToArray());
logger?.WriteLine(this.GetType().ToString(), $"Saving image to {saveImagePath}");
return filename;
}
}

View File

@ -0,0 +1,196 @@
using System.Net;
using System.Text.RegularExpressions;
using HtmlAgilityPack;
using Logging;
namespace Tranga.Connectors;
public class Manganato : Connector
{
public override string name { get; }
public Manganato(string downloadLocation, string imageCachePath, Logger? logger) : base(downloadLocation, imageCachePath, logger)
{
this.name = "Manganato";
this.downloadClient = new DownloadClient(new Dictionary<byte, int>()
{
{(byte)1, 60}
}, logger);
}
public override Publication[] GetPublications(string publicationTitle = "")
{
logger?.WriteLine(this.GetType().ToString(), $"Getting Publications (title={publicationTitle})");
string sanitizedTitle = string.Concat(Regex.Matches(publicationTitle, "[A-z]* *")).ToLower().Replace(' ', '_');
logger?.WriteLine(this.GetType().ToString(), $"Getting Publications (title={sanitizedTitle})");
string requestUrl = $"https://manganato.com/search/story/{sanitizedTitle}";
DownloadClient.RequestResult requestResult =
downloadClient.MakeRequest(requestUrl, (byte)1);
if (requestResult.statusCode != HttpStatusCode.OK)
return Array.Empty<Publication>();
return ParsePublicationsFromHtml(requestResult.result);
}
private Publication[] ParsePublicationsFromHtml(Stream html)
{
StreamReader reader = new (html);
string htmlString = reader.ReadToEnd();
HtmlDocument document = new ();
document.LoadHtml(htmlString);
IEnumerable<HtmlNode> searchResults = document.DocumentNode.Descendants("div").Where(n => n.HasClass("search-story-item"));
List<string> urls = new();
foreach (HtmlNode mangaResult in searchResults)
{
urls.Add(mangaResult.Descendants("a").First(n => n.HasClass("item-title")).GetAttributes()
.First(a => a.Name == "href").Value);
}
HashSet<Publication> ret = new();
foreach (string url in urls)
{
DownloadClient.RequestResult requestResult =
downloadClient.MakeRequest(url, (byte)1);
if (requestResult.statusCode != HttpStatusCode.OK)
return Array.Empty<Publication>();
ret.Add(ParseSinglePublicationFromHtml(requestResult.result, url.Split('/')[^1]));
}
return ret.ToArray();
}
private Publication ParseSinglePublicationFromHtml(Stream html, string publicationId)
{
StreamReader reader = new (html);
string htmlString = reader.ReadToEnd();
HtmlDocument document = new ();
document.LoadHtml(htmlString);
string status = "";
Dictionary<string, string> altTitles = new();
Dictionary<string, string>? links = null;
HashSet<string> tags = new();
string? author = null, originalLanguage = null;
int? year = DateTime.Now.Year;
HtmlNode infoNode = document.DocumentNode.Descendants("div").First(d => d.HasClass("story-info-right"));
string sortName = infoNode.Descendants("h1").First().InnerText;
HtmlNode infoTable = infoNode.Descendants().First(d => d.Name == "table");
foreach (HtmlNode row in infoTable.Descendants("tr"))
{
string key = row.SelectNodes("td").First().InnerText.ToLower();
string value = row.SelectNodes("td").Last().InnerText;
string keySanitized = string.Concat(Regex.Matches(key, "[a-z]"));
switch (keySanitized)
{
case "alternative":
string[] alts = value.Split(" ; ");
for(int i = 0; i < alts.Length; i++)
altTitles.Add(i.ToString(), alts[i]);
break;
case "authors":
author = value;
break;
case "status":
status = value;
break;
case "genres":
string[] genres = value.Split(" - ");
tags = genres.ToHashSet();
break;
default: break;
}
}
string posterUrl = document.DocumentNode.Descendants("span").First(s => s.HasClass("info-image")).Descendants("img").First()
.GetAttributes().First(a => a.Name == "src").Value;
string coverFileNameInCache = SaveCoverImageToCache(posterUrl, 1);
string description = document.DocumentNode.Descendants("div").First(d => d.HasClass("panel-story-info-description"))
.InnerText.Replace("Description :", "");
while (description.StartsWith('\n'))
description = description.Substring(1);
string yearString = document.DocumentNode.Descendants("li").Last(li => li.HasClass("a-h")).Descendants("span")
.First(s => s.HasClass("chapter-time")).InnerText;
year = Convert.ToInt32(yearString.Split(',')[^1]) + 2000;
return new Publication(sortName, author, description, altTitles, tags.ToArray(), posterUrl, coverFileNameInCache, links,
year, originalLanguage, status, publicationId);
}
public override Chapter[] GetChapters(Publication publication, string language = "")
{
logger?.WriteLine(this.GetType().ToString(), $"Getting Chapters for {publication.sortName} {publication.internalId} (language={language})");
string requestUrl = $"https://chapmanganato.com/{publication.publicationId}";
DownloadClient.RequestResult requestResult =
downloadClient.MakeRequest(requestUrl, (byte)1);
if (requestResult.statusCode != HttpStatusCode.OK)
return Array.Empty<Chapter>();
return ParseChaptersFromHtml(requestResult.result);
}
private Chapter[] ParseChaptersFromHtml(Stream html)
{
StreamReader reader = new (html);
string htmlString = reader.ReadToEnd();
HtmlDocument document = new ();
document.LoadHtml(htmlString);
List<Chapter> ret = new();
HtmlNode chapterList = document.DocumentNode.Descendants("ul").First(l => l.HasClass("row-content-chapter"));
foreach (HtmlNode chapterInfo in chapterList.Descendants("li"))
{
string fullString = chapterInfo.Descendants("a").First(d => d.HasClass("chapter-name")).InnerText;
string? volumeNumber = fullString.Contains("Vol.") ? fullString.Replace("Vol.", "").Split(' ')[0] : null;
string? chapterNumber = fullString.Split(':')[0].Split("Chapter ")[1].Replace('-','.');
string chapterName = string.Concat(fullString.Split(':')[1..]);
string url = chapterInfo.Descendants("a").First(d => d.HasClass("chapter-name"))
.GetAttributeValue("href", "");
ret.Add(new Chapter(chapterName, volumeNumber, chapterNumber, url));
}
return ret.ToArray();
}
public override void DownloadChapter(Publication publication, Chapter chapter, TrangaTask parentTask)
{
logger?.WriteLine(this.GetType().ToString(), $"Downloading Chapter-Info {publication.sortName} {publication.internalId} {chapter.volumeNumber}-{chapter.chapterNumber}");
string requestUrl = chapter.url;
DownloadClient.RequestResult requestResult =
downloadClient.MakeRequest(requestUrl, (byte)1);
if (requestResult.statusCode != HttpStatusCode.OK)
return;
string[] imageUrls = ParseImageUrlsFromHtml(requestResult.result);
string comicInfoPath = Path.GetTempFileName();
File.WriteAllText(comicInfoPath, GetComicInfoXmlString(publication, chapter, logger));
DownloadChapterImages(imageUrls, GetArchiveFilePath(publication, chapter), (byte)1, parentTask, comicInfoPath, "https://chapmanganato.com/");
}
private string[] ParseImageUrlsFromHtml(Stream html)
{
StreamReader reader = new (html);
string htmlString = reader.ReadToEnd();
HtmlDocument document = new ();
document.LoadHtml(htmlString);
List<string> ret = new();
HtmlNode imageContainer =
document.DocumentNode.Descendants("div").First(i => i.HasClass("container-chapter-reader"));
foreach(HtmlNode imageNode in imageContainer.Descendants("img"))
ret.Add(imageNode.GetAttributeValue("src", ""));
return ret.ToArray();
}
}

View File

@ -1,148 +0,0 @@
using System.Net;
using System.Net.Http.Headers;
using System.Text.Json.Nodes;
using Logging;
using Newtonsoft.Json;
using JsonSerializer = System.Text.Json.JsonSerializer;
namespace Tranga;
/// <summary>
/// Provides connectivity to Komga-API
/// Can fetch and update libraries
/// </summary>
public class Komga
{
public string baseUrl { get; }
public string auth { get; } //Base64 encoded, if you use your password everywhere, you have problems
private Logger? logger;
/// <param name="baseUrl">Base-URL of Komga instance, no trailing slashes(/)</param>
/// <param name="username">Komga Username</param>
/// <param name="password">Komga password, will be base64 encoded. yea</param>
public Komga(string baseUrl, string username, string password, Logger? logger)
{
this.baseUrl = baseUrl;
this.auth = Convert.ToBase64String(System.Text.Encoding.ASCII.GetBytes($"{username}:{password}"));
this.logger = logger;
}
/// <param name="baseUrl">Base-URL of Komga instance, no trailing slashes(/)</param>
/// <param name="auth">Base64 string of username and password (username):(password)</param>
[JsonConstructor]
public Komga(string baseUrl, string auth, Logger? logger)
{
this.baseUrl = baseUrl;
this.auth = auth;
this.logger = logger;
}
/// <summary>
/// Fetches all libraries available to the user
/// </summary>
/// <returns>Array of KomgaLibraries</returns>
public KomgaLibrary[] GetLibraries()
{
logger?.WriteLine(this.GetType().ToString(), $"Getting Libraries");
Stream data = NetClient.MakeRequest($"{baseUrl}/api/v1/libraries", auth);
if (data == Stream.Null)
{
logger?.WriteLine(this.GetType().ToString(), $"No libraries returned");
return Array.Empty<KomgaLibrary>();
}
JsonArray? result = JsonSerializer.Deserialize<JsonArray>(data);
if (result is null)
{
logger?.WriteLine(this.GetType().ToString(), $"No libraries returned");
return Array.Empty<KomgaLibrary>();
}
HashSet<KomgaLibrary> ret = new();
foreach (JsonNode? jsonNode in result)
{
var jObject = (JsonObject?)jsonNode;
string libraryId = jObject!["id"]!.GetValue<string>();
string libraryName = jObject!["name"]!.GetValue<string>();
ret.Add(new KomgaLibrary(libraryId, libraryName));
}
return ret.ToArray();
}
/// <summary>
/// Updates library with given id
/// </summary>
/// <param name="libraryId">Id of the Komga-Library</param>
/// <returns>true if successful</returns>
public bool UpdateLibrary(string libraryId)
{
logger?.WriteLine(this.GetType().ToString(), $"Updating Libraries");
return NetClient.MakePost($"{baseUrl}/api/v1/libraries/{libraryId}/scan", auth);
}
public struct KomgaLibrary
{
public string id { get; }
public string name { get; }
public KomgaLibrary(string id, string name)
{
this.id = id;
this.name = name;
}
}
private static class NetClient
{
public static Stream MakeRequest(string url, string auth)
{
HttpClientHandler clientHandler = new ();
clientHandler.ServerCertificateCustomValidationCallback = (message, cert, chain, sslPolicyErrors) => true;
HttpClient client = new(clientHandler);
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", auth);
HttpRequestMessage requestMessage = new ()
{
Method = HttpMethod.Get,
RequestUri = new Uri(url)
};
HttpResponseMessage response = client.Send(requestMessage);
Stream ret;
if (response.StatusCode is HttpStatusCode.Unauthorized)
{
ret = MakeRequest(response.RequestMessage!.RequestUri!.AbsoluteUri, auth);
}else
return response.IsSuccessStatusCode ? response.Content.ReadAsStream() : Stream.Null;
return ret;
}
public static bool MakePost(string url, string auth)
{
HttpClientHandler clientHandler = new HttpClientHandler();
clientHandler.ServerCertificateCustomValidationCallback = (message, cert, chain, sslPolicyErrors) => true;
HttpClient client = new(clientHandler)
{
DefaultRequestHeaders =
{
{ "Accept", "application/json" },
{ "Authorization", new AuthenticationHeaderValue("Basic", auth).ToString() }
}
};
HttpRequestMessage requestMessage = new HttpRequestMessage
{
Method = HttpMethod.Post,
RequestUri = new Uri(url)
};
HttpResponseMessage response = client.Send(requestMessage);
bool ret;
if (response.StatusCode is HttpStatusCode.Unauthorized)
{
ret = MakePost(response.RequestMessage!.RequestUri!.AbsoluteUri, auth);
}else
return response.IsSuccessStatusCode;
return ret;
}
}
}

123
Tranga/LibraryManager.cs Normal file
View File

@ -0,0 +1,123 @@
using System.Net;
using System.Net.Http.Headers;
using Logging;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using Tranga.LibraryManagers;
namespace Tranga;
public abstract class LibraryManager
{
public enum LibraryType : byte
{
Komga = 0,
Kavita = 1
}
public LibraryType libraryType { get; }
public string baseUrl { get; }
protected string auth { get; } //Base64 encoded, if you use your password everywhere, you have problems
protected Logger? logger;
/// <param name="baseUrl">Base-URL of Komga instance, no trailing slashes(/)</param>
/// <param name="auth">Base64 string of username and password (username):(password)</param>
/// <param name="logger"></param>
protected LibraryManager(string baseUrl, string auth, Logger? logger, LibraryType libraryType)
{
this.baseUrl = baseUrl;
this.auth = auth;
this.logger = logger;
this.libraryType = libraryType;
}
public abstract void UpdateLibrary();
public void AddLogger(Logger newLogger)
{
this.logger = newLogger;
}
protected static class NetClient
{
public static Stream MakeRequest(string url, string auth, Logger? logger)
{
HttpClientHandler clientHandler = new ();
HttpClient client = new(clientHandler);
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", auth);
HttpRequestMessage requestMessage = new ()
{
Method = HttpMethod.Get,
RequestUri = new Uri(url)
};
logger?.WriteLine("LibraryManager", $"GET {url}");
HttpResponseMessage response = client.Send(requestMessage);
logger?.WriteLine("LibraryManager", $"{(int)response.StatusCode} {response.StatusCode}: {response.ReasonPhrase}");
if(response.StatusCode is HttpStatusCode.Unauthorized && response.RequestMessage!.RequestUri!.AbsoluteUri != url)
return MakeRequest(response.RequestMessage!.RequestUri!.AbsoluteUri, auth, logger);
else if (response.IsSuccessStatusCode)
return response.Content.ReadAsStream();
else
return Stream.Null;
}
public static bool MakePost(string url, string auth, Logger? logger)
{
HttpClientHandler clientHandler = new ();
HttpClient client = new(clientHandler)
{
DefaultRequestHeaders =
{
{ "Accept", "application/json" },
{ "Authorization", new AuthenticationHeaderValue("Basic", auth).ToString() }
}
};
HttpRequestMessage requestMessage = new ()
{
Method = HttpMethod.Post,
RequestUri = new Uri(url)
};
logger?.WriteLine("LibraryManager", $"POST {url}");
HttpResponseMessage response = client.Send(requestMessage);
logger?.WriteLine("LibraryManager", $"{(int)response.StatusCode} {response.StatusCode}: {response.ReasonPhrase}");
if(response.StatusCode is HttpStatusCode.Unauthorized && response.RequestMessage!.RequestUri!.AbsoluteUri != url)
return MakePost(response.RequestMessage!.RequestUri!.AbsoluteUri, auth, logger);
else if (response.IsSuccessStatusCode)
return true;
else
return false;
}
}
public class LibraryManagerJsonConverter : JsonConverter
{
public override bool CanConvert(Type objectType)
{
return (objectType == typeof(LibraryManager));
}
public override object ReadJson(JsonReader reader, Type objectType, object? existingValue, JsonSerializer serializer)
{
JObject jo = JObject.Load(reader);
if (jo["libraryType"]!.Value<Int64>() == (Int64)LibraryType.Komga)
return jo.ToObject<Komga>(serializer)!;
if (jo["libraryType"]!.Value<Int64>() == (Int64)LibraryType.Kavita)
return jo.ToObject<Kavita>(serializer)!;
throw new Exception();
}
public override bool CanWrite => false;
/// <summary>
/// Don't call this
/// </summary>
public override void WriteJson(JsonWriter writer, object? value, JsonSerializer serializer)
{
throw new Exception("Dont call this");
}
}
}

View File

@ -0,0 +1,64 @@
using System.Text.Json;
using System.Text.Json.Nodes;
using Logging;
namespace Tranga.LibraryManagers;
public class Kavita : LibraryManager
{
public Kavita(string baseUrl, string apiKey, Logger? logger) : base(baseUrl, apiKey, logger, LibraryType.Kavita)
{
}
public override void UpdateLibrary()
{
logger?.WriteLine(this.GetType().ToString(), $"Updating Libraries");
foreach (KavitaLibrary lib in GetLibraries())
NetClient.MakePost($"{baseUrl}/api/Library/scan?libraryId={lib.id}", auth, logger);
}
/// <summary>
/// Fetches all libraries available to the user
/// </summary>
/// <returns>Array of KavitaLibrary</returns>
private IEnumerable<KavitaLibrary> GetLibraries()
{
logger?.WriteLine(this.GetType().ToString(), $"Getting Libraries");
Stream data = NetClient.MakeRequest($"{baseUrl}/api/Library", auth, logger);
if (data == Stream.Null)
{
logger?.WriteLine(this.GetType().ToString(), $"No libraries returned");
return Array.Empty<KavitaLibrary>();
}
JsonArray? result = JsonSerializer.Deserialize<JsonArray>(data);
if (result is null)
{
logger?.WriteLine(this.GetType().ToString(), $"No libraries returned");
return Array.Empty<KavitaLibrary>();
}
HashSet<KavitaLibrary> ret = new();
foreach (JsonNode? jsonNode in result)
{
var jObject = (JsonObject?)jsonNode;
string libraryId = jObject!["id"]!.GetValue<string>();
string libraryName = jObject!["name"]!.GetValue<string>();
ret.Add(new KavitaLibrary(libraryId, libraryName));
}
return ret;
}
private struct KavitaLibrary
{
public string id { get; }
public string name { get; }
public KavitaLibrary(string id, string name)
{
this.id = id;
this.name = name;
}
}
}

View File

@ -0,0 +1,75 @@
using System.Text.Json.Nodes;
using Logging;
using Newtonsoft.Json;
using JsonSerializer = System.Text.Json.JsonSerializer;
namespace Tranga.LibraryManagers;
/// <summary>
/// Provides connectivity to Komga-API
/// Can fetch and update libraries
/// </summary>
public class Komga : LibraryManager
{
public Komga(string baseUrl, string username, string password, Logger? logger)
: base(baseUrl, Convert.ToBase64String(System.Text.Encoding.ASCII.GetBytes($"{username}:{password}")), logger, LibraryType.Komga)
{
}
[JsonConstructor]
public Komga(string baseUrl, string auth, Logger? logger) : base(baseUrl, auth, logger, LibraryType.Komga)
{
}
public override void UpdateLibrary()
{
logger?.WriteLine(this.GetType().ToString(), $"Updating Libraries");
foreach (KomgaLibrary lib in GetLibraries())
NetClient.MakePost($"{baseUrl}/api/v1/libraries/{lib.id}/scan", auth, logger);
}
/// <summary>
/// Fetches all libraries available to the user
/// </summary>
/// <returns>Array of KomgaLibraries</returns>
private IEnumerable<KomgaLibrary> GetLibraries()
{
logger?.WriteLine(this.GetType().ToString(), $"Getting Libraries");
Stream data = NetClient.MakeRequest($"{baseUrl}/api/v1/libraries", auth, logger);
if (data == Stream.Null)
{
logger?.WriteLine(this.GetType().ToString(), $"No libraries returned");
return Array.Empty<KomgaLibrary>();
}
JsonArray? result = JsonSerializer.Deserialize<JsonArray>(data);
if (result is null)
{
logger?.WriteLine(this.GetType().ToString(), $"No libraries returned");
return Array.Empty<KomgaLibrary>();
}
HashSet<KomgaLibrary> ret = new();
foreach (JsonNode? jsonNode in result)
{
var jObject = (JsonObject?)jsonNode;
string libraryId = jObject!["id"]!.GetValue<string>();
string libraryName = jObject!["name"]!.GetValue<string>();
ret.Add(new KomgaLibrary(libraryId, libraryName));
}
return ret;
}
private struct KomgaLibrary
{
public string id { get; }
public string name { get; }
public KomgaLibrary(string id, string name)
{
this.id = id;
this.name = name;
}
}
}

View File

@ -1,6 +1,8 @@
using System.Text;
using System.Runtime.InteropServices;
using System.Text;
using System.Text.RegularExpressions;
using Newtonsoft.Json;
using static System.IO.UnixFileMode;
namespace Tranga;
@ -48,6 +50,26 @@ public readonly struct Publication
this.internalId = Convert.ToBase64String(Encoding.ASCII.GetBytes($"{onlyLowerLetters}{this.year}"));
}
public string CreatePublicationFolder(string downloadDirectory)
{
string publicationFolder = Path.Join(downloadDirectory, this.folderName);
if(!Directory.Exists(publicationFolder))
Directory.CreateDirectory(publicationFolder);
if(RuntimeInformation.IsOSPlatform(OSPlatform.Linux))
File.SetUnixFileMode(publicationFolder, GroupRead | GroupWrite | GroupExecute | OtherRead | OtherWrite | OtherExecute | UserRead | UserWrite | UserExecute);
return publicationFolder;
}
public void SaveSeriesInfoJson(string downloadDirectory)
{
string publicationFolder = CreatePublicationFolder(downloadDirectory);
string seriesInfoPath = Path.Join(publicationFolder, "series.json");
if(!File.Exists(seriesInfoPath))
File.WriteAllText(seriesInfoPath,this.GetSeriesInfoJson());
if(RuntimeInformation.IsOSPlatform(OSPlatform.Linux))
File.SetUnixFileMode(seriesInfoPath, GroupRead | GroupWrite | OtherRead | OtherWrite | UserRead | UserWrite);
}
/// <returns>Serialized JSON String for series.json</returns>
public string GetSeriesInfoJson()
{
@ -85,9 +107,9 @@ public readonly struct Publication
{
this.name = name;
this.year = year;
if(status == "ongoing" || status == "hiatus")
if(status.ToLower() == "ongoing" || status.ToLower() == "hiatus")
this.status = "Continuing";
else if (status == "completed" || status == "cancelled")
else if (status.ToLower() == "completed" || status.ToLower() == "cancelled")
this.status = "Ended";
else
this.status = status;

View File

@ -1,129 +0,0 @@
using Logging;
namespace Tranga;
/// <summary>
/// Executes TrangaTasks
/// Based on the TrangaTask.Task a method is called.
/// The chapterCollection is updated with new Publications/Chapters.
/// </summary>
public static class TaskExecutor
{
/// <summary>
/// Executes TrangaTask.
/// </summary>
/// <param name="taskManager">Parent</param>
/// <param name="trangaTask">Task to execute</param>
/// <param name="chapterCollection">Current chapterCollection to update</param>
/// <param name="logger"></param>
/// <exception cref="ArgumentException">Is thrown when there is no Connector available with the name of the TrangaTask.connectorName</exception>
public static void Execute(TaskManager taskManager, TrangaTask trangaTask, Logger? logger)
{
//Only execute task if it is not already being executed.
if (trangaTask.state == TrangaTask.ExecutionState.Running)
{
logger?.WriteLine("TaskExecutor", $"Task already running {trangaTask}");
return;
}
trangaTask.state = TrangaTask.ExecutionState.Running;
logger?.WriteLine("TaskExecutor", $"Starting Task {trangaTask}");
//Connector is not needed for all tasks
Connector? connector = null;
if (trangaTask.task != TrangaTask.Task.UpdateKomgaLibrary)
connector = taskManager.GetConnector(trangaTask.connectorName!);
//Call appropriate Method based on TrangaTask.Task
switch (trangaTask.task)
{
case TrangaTask.Task.DownloadNewChapters:
DownloadNewChapters(connector!, (Publication)trangaTask.publication!, trangaTask.language, ref taskManager._chapterCollection, taskManager.settings);
break;
case TrangaTask.Task.UpdateChapters:
UpdateChapters(connector!, (Publication)trangaTask.publication!, trangaTask.language, ref taskManager._chapterCollection);
break;
case TrangaTask.Task.UpdatePublications:
UpdatePublications(connector!, ref taskManager._chapterCollection);
break;
case TrangaTask.Task.UpdateKomgaLibrary:
UpdateKomgaLibrary(taskManager);
break;
}
logger?.WriteLine("TaskExecutor", $"Task finished! {trangaTask}");
trangaTask.lastExecuted = DateTime.Now;
trangaTask.state = TrangaTask.ExecutionState.Waiting;
}
/// <summary>
/// Updates all Komga-Libraries
/// </summary>
/// <param name="taskManager">Parent</param>
private static void UpdateKomgaLibrary(TaskManager taskManager)
{
if (taskManager.komga is null)
return;
Komga komga = taskManager.komga;
Komga.KomgaLibrary[] allLibraries = komga.GetLibraries();
foreach (Komga.KomgaLibrary lib in allLibraries)
komga.UpdateLibrary(lib.id);
}
/// <summary>
/// Updates the available Publications from a Connector (all of them)
/// </summary>
/// <param name="connector">Connector to receive Publications from</param>
/// <param name="chapterCollection"></param>
private static void UpdatePublications(Connector connector, ref Dictionary<Publication, List<Chapter>> chapterCollection)
{
Publication[] publications = connector.GetPublications();
foreach (Publication publication in publications)
chapterCollection.TryAdd(publication, new List<Chapter>());
}
/// <summary>
/// Checks for new Chapters and Downloads new ones.
/// If no Chapters had been downloaded previously, download also cover and create series.json
/// </summary>
/// <param name="connector">Connector to use</param>
/// <param name="publication">Publication to check</param>
/// <param name="language">Language to receive chapters for</param>
/// <param name="chapterCollection"></param>
private static void DownloadNewChapters(Connector connector, Publication publication, string language, ref Dictionary<Publication, List<Chapter>> chapterCollection, TrangaSettings settings)
{
//Check if Publication already has a Folder
string publicationFolder = Path.Join(connector.downloadLocation, publication.folderName);
if(!Directory.Exists(publicationFolder))
Directory.CreateDirectory(publicationFolder);
List<Chapter> newChapters = UpdateChapters(connector, publication, language, ref chapterCollection);
connector.CloneCoverFromCache(publication, settings);
string seriesInfoPath = Path.Join(publicationFolder, "series.json");
if(!File.Exists(seriesInfoPath))
File.WriteAllText(seriesInfoPath,publication.GetSeriesInfoJson());
foreach(Chapter newChapter in newChapters)
connector.DownloadChapter(publication, newChapter);
}
/// <summary>
/// Updates the available Chapters of a Publication
/// </summary>
/// <param name="connector">Connector to use</param>
/// <param name="publication">Publication to check</param>
/// <param name="language">Language to receive chapters for</param>
/// <param name="chapterCollection"></param>
/// <returns>List of Chapters that were previously not in collection</returns>
private static List<Chapter> UpdateChapters(Connector connector, Publication publication, string language, ref Dictionary<Publication, List<Chapter>> chapterCollection)
{
List<Chapter> newChaptersList = new();
chapterCollection.TryAdd(publication, newChaptersList); //To ensure publication is actually in collection
Chapter[] newChapters = connector.GetChapters(publication, language);
newChaptersList = newChapters.Where(nChapter => !connector.ChapterIsDownloaded(publication, nChapter)).ToList();
return newChaptersList;
}
}

View File

@ -1,6 +1,8 @@
using Logging;
using Newtonsoft.Json;
using Tranga.Connectors;
using Tranga.LibraryManagers;
using Tranga.TrangaTasks;
namespace Tranga;
@ -10,14 +12,13 @@ namespace Tranga;
/// </summary>
public class TaskManager
{
public Dictionary<Publication, List<Chapter>> _chapterCollection = new();
public Dictionary<Publication, List<Chapter>> chapterCollection = new();
private HashSet<TrangaTask> _allTasks;
private bool _continueRunning = true;
private readonly Connector[] _connectors;
private readonly Dictionary<Connector, List<TrangaTask>> _taskQueue = new();
public TrangaSettings settings { get; }
private Logger? logger { get; }
public Komga? komga => settings.komga;
/// <param name="downloadFolderPath">Local path to save data (Manga) to</param>
/// <param name="workingDirectory">Path to the working directory</param>
@ -26,19 +27,19 @@ public class TaskManager
/// <param name="komgaUsername">The Komga username</param>
/// <param name="komgaPassword">The Komga password</param>
/// <param name="logger"></param>
public TaskManager(string downloadFolderPath, string workingDirectory, string imageCachePath, string? komgaBaseUrl = null, string? komgaUsername = null, string? komgaPassword = null, Logger? logger = null)
public TaskManager(string downloadFolderPath, string workingDirectory, string imageCachePath, HashSet<LibraryManager> libraryManagers, Logger? logger = null)
{
this.logger = logger;
_allTasks = new HashSet<TrangaTask>();
Komga? newKomga = null;
if (komgaBaseUrl != null && komgaUsername != null && komgaPassword != null)
newKomga = new Komga(komgaBaseUrl, komgaUsername, komgaPassword, logger);
this.settings = new TrangaSettings(downloadFolderPath, workingDirectory, libraryManagers);
ExportDataAndSettings();
this.settings = new TrangaSettings(downloadFolderPath, workingDirectory, newKomga);
ExportData();
this._connectors = new Connector[]{ new MangaDex(downloadFolderPath, imageCachePath, logger) };
this._connectors = new Connector[]
{
new MangaDex(downloadFolderPath, imageCachePath, logger),
new Manganato(downloadFolderPath, imageCachePath, logger)
};
foreach(Connector cConnector in this._connectors)
_taskQueue.Add(cConnector, new List<TrangaTask>());
@ -46,27 +47,38 @@ public class TaskManager
taskChecker.Start();
}
public void UpdateSettings(string? downloadLocation, string? komgaUrl, string? komgaAuth)
public void UpdateSettings(string? downloadLocation, string? komgaUrl, string? komgaAuth, string? kavitaUrl, string? kavitaApiKey)
{
Komga? komga = null;
if (komgaUrl is not null && komgaAuth is not null)
komga = new Komga(komgaUrl, komgaAuth, null);
settings.downloadLocation = downloadLocation ?? settings.downloadLocation;
settings.komga = komga ?? komga;
ExportData();
if (komgaUrl is not null && komgaAuth is not null && komgaUrl.Length > 0 && komgaAuth.Length > 0)
{
settings.libraryManagers.RemoveWhere(lm => lm.GetType() == typeof(Komga));
settings.libraryManagers.Add(new Komga(komgaUrl, komgaAuth, logger));
}
if (kavitaUrl is not null && kavitaApiKey is not null && kavitaUrl.Length > 0 && kavitaApiKey.Length > 0)
{
settings.libraryManagers.RemoveWhere(lm => lm.GetType() == typeof(Kavita));
settings.libraryManagers.Add(new Kavita(kavitaUrl, kavitaApiKey, logger));
}
if (downloadLocation is not null && downloadLocation.Length > 0)
settings.downloadLocation = downloadLocation;
ExportDataAndSettings();
}
public TaskManager(TrangaSettings settings, Logger? logger = null)
{
this.logger = logger;
this._connectors = new Connector[]{ new MangaDex(settings.downloadLocation, settings.coverImageCache, logger) };
this._connectors = new Connector[]
{
new MangaDex(settings.downloadLocation, settings.coverImageCache, logger),
new Manganato(settings.downloadLocation, settings.coverImageCache, logger)
};
foreach(Connector cConnector in this._connectors)
_taskQueue.Add(cConnector, new List<TrangaTask>());
_allTasks = new HashSet<TrangaTask>();
this.settings = settings;
ImportData();
ExportData();
ExportDataAndSettings();
Thread taskChecker = new(TaskCheckerThread);
taskChecker.Start();
}
@ -84,7 +96,7 @@ public class TaskManager
foreach (KeyValuePair<Connector, List<TrangaTask>> connectorTaskQueue in _taskQueue)
{
if(connectorTaskQueue.Value.RemoveAll(task => task.state == TrangaTask.ExecutionState.Waiting) > 0)
ExportData();
ExportDataAndSettings();
if (connectorTaskQueue.Value.Count > 0 && connectorTaskQueue.Value.All(task => task.state is TrangaTask.ExecutionState.Enqueued))
ExecuteTaskNow(connectorTaskQueue.Value.First());
@ -95,7 +107,7 @@ public class TaskManager
foreach (TrangaTask task in _allTasks.Where(aTask => aTask.ShouldExecute()))
{
task.state = TrangaTask.ExecutionState.Enqueued;
if(task.connectorName is null)
if(task.task == TrangaTask.Task.UpdateLibraries)
ExecuteTaskNow(task);
else
{
@ -116,10 +128,9 @@ public class TaskManager
if (!this._allTasks.Contains(task))
return;
logger?.WriteLine(this.GetType().ToString(), $"Forcing Execution: {task}");
Task t = new Task(() =>
Task t = new(() =>
{
TaskExecutor.Execute(this, task, logger);
task.Execute(this, this.logger);
});
t.Start();
}
@ -138,41 +149,41 @@ public class TaskManager
{
logger?.WriteLine(this.GetType().ToString(), $"Adding new Task {task} {connectorName} {publication?.sortName}");
TrangaTask newTask;
if (task == TrangaTask.Task.UpdateKomgaLibrary)
TrangaTask? newTask = null;
if (task == TrangaTask.Task.UpdateLibraries)
{
newTask = new TrangaTask(task, null, null, reoccurrence);
newTask = new UpdateLibrariesTask(task, reoccurrence);
logger?.WriteLine(this.GetType().ToString(), $"Removing old {task}-Task.");
//Only one UpdateKomgaLibrary Task
_allTasks.RemoveWhere(trangaTask => trangaTask.task is TrangaTask.Task.UpdateKomgaLibrary);
_allTasks.RemoveWhere(trangaTask => trangaTask.task is TrangaTask.Task.UpdateLibraries);
_allTasks.Add(newTask);
}
else
logger?.WriteLine(this.GetType().ToString(), $"Added new Task {newTask}");
}else if (task == TrangaTask.Task.DownloadNewChapters)
{
if(connectorName is null)
throw new ArgumentException($"connectorName can not be null for task {task}");
//Get appropriate Connector from available Connectors for TrangaTask
Connector? connector = _connectors.FirstOrDefault(c => c.name == connectorName);
if (connector is null)
throw new ArgumentException($"Connector {connectorName} is not a known connector.");
if (connectorName is null)
throw new ArgumentException($"connectorName can not be null for task {task}");
newTask = new TrangaTask(task, connector.name, publication, reoccurrence, language);
if (publication is null)
throw new ArgumentException($"publication can not be null for task {task}");
Publication pub = (Publication)publication;
newTask = new DownloadNewChaptersTask(task, connector!.name, pub, reoccurrence, language);
//Check if same task already exists
if (!_allTasks.Any(trangaTask => trangaTask.task == task && trangaTask.connectorName == connector.name &&
trangaTask.publication?.internalId == publication?.internalId))
if (!_allTasks.Any(trangaTask =>
trangaTask.task == task && trangaTask.publication?.internalId == pub.internalId &&
trangaTask.connectorName == connector.name))
{
if(task != TrangaTask.Task.UpdatePublications)
_chapterCollection.TryAdd((Publication)publication!, new List<Chapter>());
_allTasks.Add(newTask);
logger?.WriteLine(this.GetType().ToString(), $"Added new Task {newTask}");
}
else
logger?.WriteLine(this.GetType().ToString(), $"Publication already exists {publication?.internalId}");
logger?.WriteLine(this.GetType().ToString(), $"Task already exists {newTask}");
}
logger?.WriteLine(this.GetType().ToString(), $"Added new Task {newTask.ToString()}");
ExportData();
ExportDataAndSettings();
if (newTask is null)
throw new Exception("Invalid path");
return newTask;
}
@ -185,9 +196,9 @@ public class TaskManager
public void DeleteTask(TrangaTask.Task task, string? connectorName, Publication? publication)
{
logger?.WriteLine(this.GetType().ToString(), $"Removing Task {task} {publication?.sortName}");
if (task == TrangaTask.Task.UpdateKomgaLibrary)
if (task == TrangaTask.Task.UpdateLibraries)
{
_allTasks.RemoveWhere(uTask => uTask.task == TrangaTask.Task.UpdateKomgaLibrary);
_allTasks.RemoveWhere(uTask => uTask.task == TrangaTask.Task.UpdateLibraries);
logger?.WriteLine(this.GetType().ToString(), $"Removed Task {task} from all Tasks.");
}
else if (connectorName is null)
@ -208,7 +219,7 @@ public class TaskManager
else
logger?.WriteLine(this.GetType().ToString(), $"No Task {task} {publication?.sortName} {publication?.internalId} could be found.");
}
ExportData();
ExportDataAndSettings();
}
/// <summary>
@ -252,8 +263,8 @@ public class TaskManager
Publication[] ret = connector.GetPublications(title ?? "");
foreach (Publication publication in ret)
{
if(!_chapterCollection.Any(pub => pub.Key.sortName == publication.sortName))
this._chapterCollection.TryAdd(publication, new List<Chapter>());
if(!chapterCollection.Any(pub => pub.Key.sortName == publication.sortName))
this.chapterCollection.TryAdd(publication, new List<Chapter>());
}
return ret;
}
@ -261,7 +272,7 @@ public class TaskManager
/// <returns>All added Publications</returns>
public Publication[] GetAllPublications()
{
return this._chapterCollection.Keys.ToArray();
return this.chapterCollection.Keys.ToArray();
}
/// <summary>
@ -276,7 +287,7 @@ public class TaskManager
Connector? ret = this._connectors.FirstOrDefault(connector => connector.name == connectorName);
if (ret is null)
throw new Exception($"Connector {connectorName} is not an available Connector.");
return (Connector)ret!;
return ret;
}
/// <summary>
@ -287,7 +298,7 @@ public class TaskManager
{
logger?.WriteLine(this.GetType().ToString(), $"Shutting down (forced={force})");
_continueRunning = false;
ExportData();
ExportDataAndSettings();
if(force)
Environment.Exit(_allTasks.Count(task => task.state is TrangaTask.ExecutionState.Enqueued or TrangaTask.ExecutionState.Running));
@ -307,7 +318,7 @@ public class TaskManager
{
logger?.WriteLine(this.GetType().ToString(), $"Importing tasks from {settings.tasksFilePath}");
buffer = File.ReadAllText(settings.tasksFilePath);
this._allTasks = JsonConvert.DeserializeObject<HashSet<TrangaTask>>(buffer)!;
this._allTasks = JsonConvert.DeserializeObject<HashSet<TrangaTask>>(buffer, new JsonSerializerSettings() { Converters = { new TrangaTask.TrangaTaskJsonConverter() } })!;
}
if (File.Exists(settings.knownPublicationsPath))
@ -316,14 +327,14 @@ public class TaskManager
buffer = File.ReadAllText(settings.knownPublicationsPath);
Publication[] publications = JsonConvert.DeserializeObject<Publication[]>(buffer)!;
foreach (Publication publication in publications)
this._chapterCollection.TryAdd(publication, new List<Chapter>());
this.chapterCollection.TryAdd(publication, new List<Chapter>());
}
}
/// <summary>
/// Exports data (settings, tasks) to file
/// </summary>
private void ExportData()
private void ExportDataAndSettings()
{
logger?.WriteLine(this.GetType().ToString(), $"Exporting settings to {settings.settingsFilePath}");
File.WriteAllText(settings.settingsFilePath, JsonConvert.SerializeObject(settings));
@ -332,7 +343,7 @@ public class TaskManager
File.WriteAllText(settings.tasksFilePath, JsonConvert.SerializeObject(this._allTasks));
logger?.WriteLine(this.GetType().ToString(), $"Exporting known publications to {settings.knownPublicationsPath}");
File.WriteAllText(settings.knownPublicationsPath, JsonConvert.SerializeObject(this._chapterCollection.Keys.ToArray()));
File.WriteAllText(settings.knownPublicationsPath, JsonConvert.SerializeObject(this.chapterCollection.Keys.ToArray()));
}

View File

@ -7,6 +7,7 @@
</PropertyGroup>
<ItemGroup>
<PackageReference Include="HtmlAgilityPack" Version="1.11.46" />
<PackageReference Include="Newtonsoft.Json" Version="13.0.3" />
</ItemGroup>

View File

@ -1,4 +1,6 @@
using Newtonsoft.Json;
using Logging;
using Newtonsoft.Json;
using Tranga.LibraryManagers;
namespace Tranga;
@ -10,22 +12,27 @@ public class TrangaSettings
[JsonIgnore]public string tasksFilePath => Path.Join(workingDirectory, "tasks.json");
[JsonIgnore]public string knownPublicationsPath => Path.Join(workingDirectory, "knownPublications.json");
[JsonIgnore] public string coverImageCache => Path.Join(workingDirectory, "imageCache");
public Komga? komga { get; set; }
public HashSet<LibraryManager> libraryManagers { get; }
public TrangaSettings(string downloadLocation, string workingDirectory, Komga? komga)
public TrangaSettings(string downloadLocation, string workingDirectory, HashSet<LibraryManager> libraryManagers)
{
if (downloadLocation.Length < 1 || workingDirectory.Length < 1)
throw new ArgumentException("Download-location and working-directory paths can not be empty!");
this.workingDirectory = workingDirectory;
this.downloadLocation = downloadLocation;
this.komga = komga;
this.libraryManagers = libraryManagers;
}
public static TrangaSettings LoadSettings(string importFilePath)
public static TrangaSettings LoadSettings(string importFilePath, Logger? logger)
{
if (!File.Exists(importFilePath))
return new TrangaSettings(Path.Join(Directory.GetCurrentDirectory(), "Downloads"), Directory.GetCurrentDirectory(), null);
return new TrangaSettings(Path.Join(Directory.GetCurrentDirectory(), "Downloads"), Directory.GetCurrentDirectory(), new HashSet<LibraryManager>());
string toRead = File.ReadAllText(importFilePath);
TrangaSettings settings = JsonConvert.DeserializeObject<TrangaSettings>(toRead)!;
TrangaSettings settings = JsonConvert.DeserializeObject<TrangaSettings>(toRead, new JsonSerializerSettings() { Converters = { new LibraryManager.LibraryManagerJsonConverter()} })!;
if(logger is not null)
foreach(LibraryManager lm in settings.libraryManagers)
lm.AddLogger(logger);
return settings;
}

View File

@ -1,11 +1,14 @@
using Newtonsoft.Json;
using Logging;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using Tranga.TrangaTasks;
namespace Tranga;
/// <summary>
/// Stores information on Task
/// Stores information on Task, when implementing new Tasks also update the serializer
/// </summary>
public class TrangaTask
public abstract class TrangaTask
{
// ReSharper disable once CommentTypo ...Tell me why!
// ReSharper disable once MemberCanBePrivate.Global I want it thaaat way
@ -14,8 +17,11 @@ public class TrangaTask
public string? connectorName { get; }
public Task task { get; }
public Publication? publication { get; }
public string language { get; }
public string? language { get; }
[JsonIgnore]public ExecutionState state { get; set; }
[JsonIgnore] public float progress => (tasksFinished != 0f ? tasksFinished / tasksCount : 0f);
[JsonIgnore]public float tasksCount { get; set; }
[JsonIgnore]public float tasksFinished { get; set; }
public enum ExecutionState
{
@ -24,20 +30,39 @@ public class TrangaTask
Running
};
public TrangaTask(Task task, string? connectorName, Publication? publication, TimeSpan reoccurrence, string language = "")
protected TrangaTask(Task task, string? connectorName, Publication? publication, TimeSpan reoccurrence, string? language = null)
{
if(task != Task.UpdateKomgaLibrary && connectorName is null)
throw new ArgumentException($"connectorName can not be null for task {task}");
if (publication is null && task != Task.UpdatePublications && task != Task.UpdateKomgaLibrary)
throw new ArgumentException($"Publication can not be null for task {task}");
this.publication = publication;
this.reoccurrence = reoccurrence;
this.lastExecuted = DateTime.Now.Subtract(reoccurrence);
this.connectorName = connectorName;
this.task = task;
this.language = language;
this.tasksCount = 1;
this.tasksFinished = 0;
}
/// <summary>
/// BL for concrete Tasks
/// </summary>
/// <param name="taskManager"></param>
/// <param name="logger"></param>
protected abstract void ExecuteTask(TaskManager taskManager, Logger? logger);
/// <summary>
/// Execute the task
/// </summary>
/// <param name="taskManager">Should be the parent taskManager</param>
/// <param name="logger"></param>
public void Execute(TaskManager taskManager, Logger? logger)
{
logger?.WriteLine(this.GetType().ToString(), $"Executing Task {this}");
this.state = ExecutionState.Running;
ExecuteTask(taskManager, logger);
this.lastExecuted = DateTime.Now;
this.state = ExecutionState.Waiting;
logger?.WriteLine(this.GetType().ToString(), $"Finished Executing Task {this}");
}
/// <returns>True if elapsed time since last execution is greater than set interval</returns>
@ -46,16 +71,44 @@ public class TrangaTask
return DateTime.Now.Subtract(this.lastExecuted) > reoccurrence && state is ExecutionState.Waiting;
}
public enum Task
public enum Task : byte
{
UpdatePublications,
UpdateChapters,
DownloadNewChapters,
UpdateKomgaLibrary
DownloadNewChapters = 2,
UpdateLibraries = 3
}
public override string ToString()
{
return $"{task}, {lastExecuted}, {reoccurrence}, {state} {(connectorName is not null ? $", {connectorName}" : "" )} {(publication is not null ? $", {publication?.sortName}": "")}";
return $"{task}, {lastExecuted}, {reoccurrence}, {state} {(connectorName is not null ? $", {connectorName}" : "" )} {(publication is not null ? $", {progress:00.00}%" : "")} {(publication is not null ? $", {publication?.sortName}" : "")}";
}
public class TrangaTaskJsonConverter : JsonConverter
{
public override bool CanConvert(Type objectType)
{
return (objectType == typeof(TrangaTask));
}
public override object ReadJson(JsonReader reader, Type objectType, object? existingValue, JsonSerializer serializer)
{
JObject jo = JObject.Load(reader);
if (jo["task"]!.Value<Int64>() == (Int64)Task.DownloadNewChapters)
return jo.ToObject<DownloadNewChaptersTask>(serializer)!;
if (jo["task"]!.Value<Int64>() == (Int64)Task.UpdateLibraries)
return jo.ToObject<UpdateLibrariesTask>(serializer)!;
throw new Exception();
}
public override bool CanWrite => false;
/// <summary>
/// Don't call this
/// </summary>
public override void WriteJson(JsonWriter writer, object? value, JsonSerializer serializer)
{
throw new Exception("Dont call this");
}
}
}

View File

@ -0,0 +1,47 @@
using Logging;
namespace Tranga.TrangaTasks;
public class DownloadNewChaptersTask : TrangaTask
{
public DownloadNewChaptersTask(Task task, string connectorName, Publication publication, TimeSpan reoccurrence, string language = "en") : base(task, connectorName, publication, reoccurrence, language)
{
}
protected override void ExecuteTask(TaskManager taskManager, Logger? logger)
{
Publication pub = (Publication)this.publication!;
Connector connector = taskManager.GetConnector(this.connectorName);
//Check if Publication already has a Folder
pub.CreatePublicationFolder(taskManager.settings.downloadLocation);
List<Chapter> newChapters = UpdateChapters(connector, pub, language!, ref taskManager.chapterCollection);
this.tasksCount = newChapters.Count;
connector.CopyCoverFromCacheToDownloadLocation(pub, taskManager.settings);
pub.SaveSeriesInfoJson(connector.downloadLocation);
foreach(Chapter newChapter in newChapters)
connector.DownloadChapter(pub, newChapter, this);
}
/// <summary>
/// Updates the available Chapters of a Publication
/// </summary>
/// <param name="connector">Connector to use</param>
/// <param name="publication">Publication to check</param>
/// <param name="language">Language to receive chapters for</param>
/// <param name="chapterCollection"></param>
/// <returns>List of Chapters that were previously not in collection</returns>
private static List<Chapter> UpdateChapters(Connector connector, Publication publication, string language, ref Dictionary<Publication, List<Chapter>> chapterCollection)
{
List<Chapter> newChaptersList = new();
chapterCollection.TryAdd(publication, newChaptersList); //To ensure publication is actually in collection
Chapter[] newChapters = connector.GetChapters(publication, language);
newChaptersList = newChapters.Where(nChapter => !connector.CheckChapterIsDownloaded(publication, nChapter)).ToList();
return newChaptersList;
}
}

View File

@ -0,0 +1,16 @@
using Logging;
namespace Tranga.TrangaTasks;
public class UpdateLibrariesTask : TrangaTask
{
public UpdateLibrariesTask(Task task, TimeSpan reoccurrence) : base(task, null, null, reoccurrence)
{
}
protected override void ExecuteTask(TaskManager taskManager, Logger? logger)
{
foreach(LibraryManager lm in taskManager.settings.libraryManagers)
lm.UpdateLibrary();
}
}

View File

@ -84,7 +84,7 @@ async function GetSettings(){
}
async function GetKomgaTask(){
var uri = apiUri + "/Tasks/Get?taskType=UpdateKomgaLibrary";
var uri = apiUri + "/Tasks/Get?taskType=UpdateLibraries";
let json = await GetData(uri);
return json;
}

View File

@ -4,17 +4,18 @@
<meta charset="UTF-8">
<title>Tranga</title>
<link rel="stylesheet" href="style.css">
<link rel="icon" type="image/x-icon" href="favicon.ico">
</head>
<body>
<wrapper>
<topbar>
<titlebox>
<img src="media/blahaj.png">
<img alt="website image is Blahaj" src="media/blahaj.png">
<span>Tranga</span>
</titlebox>
<spacer></spacer>
<searchdiv>
<input id="searchbox" placeholder="Filter" type="text">
<label for="searchbox"></label><input id="searchbox" placeholder="Filter" type="text">
</searchdiv>
<img id="settingscog" src="media/settings-cogwheel.svg" height="100%" alt="settingscog">
</topbar>
@ -60,6 +61,7 @@
<img id="pubviewcover" src="media/cover.jpg" alt="cover">
<publication-information>
<publication-name id="publicationViewerName">Tensei Pandemic</publication-name>
<publication-tags id="publicationViewerTags"></publication-tags>
<publication-author id="publicationViewerAuthor">Imamura Hinata</publication-author>
<publication-description id="publicationViewerDescription">Imamura Hinata is a high school boy with a cute appearance.
Since his trauma with the first love, he wanted to be more manly than anybody else. But one day he woke up to something different…
@ -92,9 +94,17 @@
<label for="komgaUrl"></label><input placeholder="URL" id="komgaUrl" type="text">
<label for="komgaUsername"></label><input placeholder="Username" id="komgaUsername" type="text">
<label for="komgaPassword"></label><input placeholder="Password" id="komgaPassword" type="password">
<label for="komgaUpdateTime" style="margin-right: 5px;">Update Time</label><input id="komgaUpdateTime" type="time" value="00:01:00" step="10">
<input type="submit" value="Update" onclick="UpdateKomgaSettings()">
</komga-settings>
<kavita-settings>
<span class="title">Kavita</span>
<div>Configured: <span id="kavitaConfigured">✅❌</span></div>
<label for="kavitaUrl"></label><input placeholder="URL" id="kavitaUrl" type="text">
<label for="kavitaApiKey"></label><input placeholder="API-Key" id="kavitaApiKey" type="text">
</kavita-settings>
<div>
<label for="libraryUpdateTime" style="margin-right: 5px;">Update Time</label><input id="libraryUpdateTime" type="time" value="00:01:00" step="10">
<input type="submit" value="Update" onclick="UpdateLibrarySettings()">
</div>
</settings>
</popup>
</viewport>

View File

@ -2,6 +2,7 @@
let tasks = [];
let toEditId;
const searchBox = document.querySelector("#searchbox");
const searchPublicationQuery = document.querySelector("#searchPublicationQuery");
const selectPublication = document.querySelector("#taskSelectOutput");
const connectorSelect = document.querySelector("#connectors");
@ -14,6 +15,7 @@ const publicationViewerPopup = document.querySelector("#publicationViewerPopup")
const publicationViewerWindow = document.querySelector("publication-viewer");
const publicationViewerDescription = document.querySelector("#publicationViewerDescription");
const publicationViewerName = document.querySelector("#publicationViewerName");
const publicationViewerTags = document.querySelector("#publicationViewerTags");
const publicationViewerAuthor = document.querySelector("#publicationViewerAuthor");
const pubviewcover = document.querySelector("#pubviewcover");
const publicationDelete = document.querySelector("publication-delete");
@ -24,8 +26,11 @@ const settingDownloadLocation = document.querySelector("#downloadLocation");
const settingKomgaUrl = document.querySelector("#komgaUrl");
const settingKomgaUser = document.querySelector("#komgaUsername");
const settingKomgaPass = document.querySelector("#komgaPassword");
const settingKomgaTime = document.querySelector("#komgaUpdateTime");
const settingKavitaUrl = document.querySelector("#kavitaUrl");
const settingKavitaApi = document.querySelector("#kavitaApiKey");
const libraryUpdateTime = document.querySelector("#libraryUpdateTime");
const settingKomgaConfigured = document.querySelector("#komgaConfigured");
const settingKavitaConfigured = document.querySelector("#kavitaConfigured");
const settingApiUri = document.querySelector("#settingApiUri");
const tagTasksRunning = document.querySelector("#tasksRunningTag");
const tagTasksQueued = document.querySelector("#tasksQueuedTag");
@ -33,6 +38,7 @@ const tagTasksTotal = document.querySelector("#totalTasksTag");
const tagTaskPopup = document.querySelector("footer-tag-popup");
const tagTasksPopupContent = document.querySelector("footer-tag-content");
searchbox.addEventListener("keyup", (event) => FilterResults());
settingsCog.addEventListener("click", () => OpenSettings());
document.querySelector("#blurBackgroundSettingsPopup").addEventListener("click", () => HideSettings());
closetaskpopup.addEventListener("click", () => HideAddTaskPopup());
@ -78,6 +84,11 @@ function NewSearch(){
selectRecurrence.disabled = true;
connectorSelect.disabled = true;
searchPublicationQuery.disabled = true;
//Waitcursor
document.body.style.cursor = "wait";
selectRecurrence.style.cursor = "wait";
connectorSelect.style.cursor = "wait";
searchPublicationQuery.style.cursor = "wait";
//Empty previous results
selectPublication.replaceChildren();
@ -96,6 +107,11 @@ function NewSearch(){
selectRecurrence.disabled = false;
connectorSelect.disabled = false;
searchPublicationQuery.disabled = false;
//Cursor
document.body.style.cursor = "initial";
selectRecurrence.style.cursor = "initial";
connectorSelect.style.cursor = "initial";
searchPublicationQuery.style.cursor = "initial";
});
}
@ -169,6 +185,7 @@ function ShowPublicationViewerWindow(publicationId, event, add){
//Edit information inside the window
var publication = publications.filter(pub => pub.internalId === publicationId)[0];
publicationViewerName.innerText = publication.sortName;
publicationViewerTags.innerText = publication.tags.join(", ");
publicationViewerDescription.innerText = publication.description;
publicationViewerAuthor.innerText = publication.author;
pubviewcover.src = `imageCache/${publication.coverFileNameInCache}`;
@ -225,35 +242,50 @@ function GetSettingsClick(){
settingKomgaUrl.value = "";
settingKomgaUser.value = "";
settingKomgaPass.value = "";
settingKavitaUrl.value = "";
settingKavitaApi.value = "";
settingKomgaConfigured.innerText = "❌";
settingKavitaConfigured.innerText = "❌";
settingApiUri.placeholder = apiUri;
GetSettings().then(json => {
settingDownloadLocation.innerText = json.downloadLocation;
if(json.komga != null)
settingKomgaUrl.placeholder = json.komga.baseUrl;
json.libraryManagers.forEach(lm => {
if(lm.libraryType == 0){
settingKomgaUrl.placeholder = lm.baseUrl;
settingKomgaUser.placeholder = "Configured";
settingKomgaPass.placeholder = "***";
settingKomgaConfigured.innerText = "✅";
} else if(libraryType == 1){
settingKavitaUrl.placeholder = lm.baseUrl;
settingKavitaApi.placeholder = "***";
settingKavitaConfigured.innerText = "✅";
}
});
});
GetKomgaTask().then(json => {
settingKomgaTime.value = json[0].reoccurrence;
if(json.length > 0)
settingKomgaConfigured.innerText = "✅";
else
settingKomgaConfigured.innerText = "❌";
libraryUpdateTime.value = json[0].reoccurrence;
});
}
function UpdateKomgaSettings(){
function UpdateLibrarySettings(){
if(settingKomgaUser.value != "" && settingKomgaPass != ""){
var auth = utf8_to_b64(`${settingKomgaUser.value}:${settingKomgaPass.value}`);
console.log(auth);
if(settingKomgaUrl.value != "")
UpdateSettings("", settingKomgaUrl.value, auth);
UpdateSettings("", settingKomgaUrl.value, auth, "", "");
else
UpdateSettings("", settingKomgaUrl.placeholder, auth);
UpdateSettings("", settingKomgaUrl.placeholder, auth, "", "");
}
CreateTask("UpdateKomgaLibrary", settingKomgaTime.value, "","","");
if(settingKavitaUrl.value != "" && settingKavitaApi != ""){
UpdateSettings("", "", "", settingKavitaUrl.value, settingKavitaApi.value);
}
CreateTask("UpdateLibraries", libraryUpdateTime.value, "","","");
setTimeout(() => GetSettingsClick(), 100);
}
@ -296,7 +328,6 @@ function ShowQueuedTasks(event){
}
});
}
function ShowAllTasks(event){
GetDownloadTasks()
.then(json => {
@ -317,6 +348,30 @@ function CloseTasksPopup(){
tagTaskPopup.style.display = "none";
}
function FilterResults(){
if(searchBox.value.length > 0){
tasksContent.childNodes.forEach(publication => {
publication.childNodes.forEach(item => {
if(item.nodeName.toLowerCase() == "publication-information"){
item.childNodes.forEach(information => {
if(information.nodeName.toLowerCase() == "publication-name"){
if(!information.textContent.toLowerCase().includes(searchBox.value.toLowerCase())){
publication.style.display = "none";
}else{
publication.style.display = "initial";
}
}
});
}
});
});
}else{
tasksContent.childNodes.forEach(publication => publication.style.display = "initial");
}
}
//Resets the tasks shown
ResetContent();
//Get Tasks and show them

View File

@ -53,6 +53,7 @@ titlebox {
}
titlebox span{
cursor: default;
font-size: 24pt;
font-weight: bold;
background: linear-gradient(150deg, var(--primary-color), var(--accent-color));
@ -64,6 +65,7 @@ titlebox span{
titlebox img {
height: 100%;
margin-right: 10px;
cursor: grab;
}
spacer{
@ -119,6 +121,7 @@ footer > div {
flex-direction: row;
flex-wrap: nowrap;
align-items: center;
cursor: pointer;
}
footer > div > *{
@ -130,6 +133,7 @@ footer > div > *{
flex-grow: 1;
text-align: right;
margin-right: 20px;
cursor: url("media/blahaj.png"), grab;
}
content {
@ -151,7 +155,7 @@ settings {
z-index: 10;
position: absolute;
left: 25%;
top: 25%;
top: 100px;
border-radius: 5px;
padding: 10px 0;
}
@ -376,14 +380,13 @@ addtask-settings addtask-setting{
publication-viewer{
display: block;
width: 450px;
height: 300px;
position: absolute;
top: 200px;
left: 400px;
background-color: var(--accent-color);
border-radius: 5px;
overflow: hidden;
padding: 30px;
padding: 15px;
}
publication-viewer::after{
@ -391,7 +394,8 @@ publication-viewer::after{
position: absolute;
left: 0; top: 0;
border-radius: 5px;
width: 100%; height: 100%;
width: 100%;
height: 100%;
background: rgba(0,0,0,0.8);
backdrop-filter: blur(3px);
}
@ -407,16 +411,32 @@ publication-viewer img {
z-index: 0;
}
publication-viewer publication-information publication-name{
publication-viewer publication-information > * {
margin: 5px 0;
}
publication-viewer publication-information publication-author {
margin: 5px 0;
publication-viewer publication-information publication-name {
width: initial;
overflow-x: scroll;
white-space: nowrap;
scrollbar-width: none;
}
publication-viewer publication-information publication-tags::before {
content: "Tags";
display: block;
font-weight: bolder;
}
publication-viewer publication-information publication-tags {
overflow-x: scroll;
white-space: nowrap;
scrollbar-width: none;
}
publication-viewer publication-information publication-author::before {
content: "Author: ";
font-weight: bolder;
}
publication-viewer publication-information publication-description::before {
@ -428,24 +448,22 @@ publication-viewer publication-information publication-description::before {
publication-viewer publication-information publication-description {
font-size: 12pt;
margin: 5px 0;
max-height: 200px;
height: 145px;
overflow-x: scroll;
}
publication-viewer publication-information publication-interactions {
display: flex;
flex-direction: row;
position: absolute;
justify-content: end;
align-items: start;
bottom: 0;
left: 0;
width: 100%;
}
publication-viewer publication-information publication-interactions > * {
margin: 0 10px 10px 10px;
margin: 0 10px;
font-size: 16pt;
cursor: pointer;
}
publication-viewer publication-information publication-interactions publication-starttask {

View File

@ -1,4 +1,5 @@
services:
version: '3'
services:
tranga-api:
image: glax/tranga-api:latest
container_name: tranga-api
@ -17,3 +18,4 @@
- 9555:80
depends_on:
- tranga-api
restart: unless-stopped