social.sokoll.com

Search

Items tagged with: translation

"Towards fully automated manga translation." "Due to the high cost of translation, most comics have not been translated and are only available in their domestic markets. What if all comics could be immediately translated into any language?"

"What makes the translation of comics difficult?" Utterances by a character are divided up into multiple bubbles, interlaced with bubbles from other characters. They are not necessarily aligned in a straightforward way such as from left to right or right to left or top to bottom. It is necessary to figure out the visual structure just to figure out what the correct order is to arrange the bubbles. Not only that, but the meaning of the pictures has to be deciphered to do a correct translation, for example, some Japanese words can be translated into both "he", "him", "she", or "her", and without understanding the pictures, these ambiguities can't be resolved.

So what these researchers did was make a manga translation dataset which is a massive collection of "parallel corpus", that is, manga that have already been translated between languages, with the translated versions paired up with the original, that an AI system can train from, as well as a system for extracting pictures and text from the corpus. From this they built a "multimodal context-aware translation system" that was able to fully automate the manga translation process. By "multimodal", they mean the same system takes input in the form of text and pictures (so two "modes"), and by "context-aware", they mean the language translation takes into account the context provided by the pictures.

The translation system receives a manga image and a set of texts. The image represents a whole page, not just a single frame, and the system will figure out whether all the frames represent the same scene, or whether there are multiple scenes. Each of the texts has a "bounding box" that indicates where it came from in the image. A convolutional neural network called Faster R-CNN is used to determine what scene or scenes are in the frames.

Next the reading order is determined. This is first estimated by a simple algorithm that assumes frames are in rows, and within each row, frames are in columns, with a right-to-left (for Japanese), then top-to-bottom reading order. This gets the reading order about 92% correct. The remaining 8% irregular cases are handled by a supervised learning algorithm.

Next a system called illustration2vec is used to create a vector (this is the "embedding" idea though they don't use the word) describing the scene. The output of this is tags like "boy", "girl".

To do the actual translation, the input text is all concatenated together, translated all at once, and then broken back apart into the proper speech bubbles on the other end. To make this work, though, tags from illustration2vec are inserted into the text, which provide context to the translation system.

All of this glosses over additional complex issues that the researchers had to solve to make this work, such as identifying where text is in an image, figure out what style the text is written in, since text in manga is written in various styles, and extract the text.

Towards Fully Automated Manga Translation

#solidstatelife #ai #computervision #nlp #translation #embeddings #manga
 
"Towards fully automated manga translation." "Due to the high cost of translation, most comics have not been translated and are only available in their domestic markets. What if all comics could be immediately translated into any language?"

"What makes the translation of comics difficult?" Utterances by a character are divided up into multiple bubbles, interlaced with bubbles from other characters. They are not necessarily aligned in a straightforward way such as from left to right or right to left or top to bottom. It is necessary to figure out the visual structure just to figure out what the correct order is to arrange the bubbles. Not only that, but the meaning of the pictures has to be deciphered to do a correct translation, for example, some Japanese words can be translated into both "he", "him", "she", or "her", and without understanding the pictures, these ambiguities can't be resolved.

So what these researchers did was make a manga translation dataset which is a massive collection of "parallel corpus", that is, manga that have already been translated between languages, with the translated versions paired up with the original, that an AI system can train from, as well as a system for extracting pictures and text from the corpus. From this they built a "multimodal context-aware translation system" that was able to fully automate the manga translation process. By "multimodal", they mean the same system takes input in the form of text and pictures (so two "modes"), and by "context-aware", they mean the language translation takes into account the context provided by the pictures.

The translation system receives a manga image and a set of texts. The image represents a whole page, not just a single frame, and the system will figure out whether all the frames represent the same scene, or whether there are multiple scenes. Each of the texts has a "bounding box" that indicates where it came from in the image. A convolutional neural network called Faster R-CNN is used to determine what scene or scenes are in the frames.

Next the reading order is determined. This is first estimated by a simple algorithm that assumes frames are in rows, and within each row, frames are in columns, with a right-to-left (for Japanese), then top-to-bottom reading order. This gets the reading order about 92% correct. The remaining 8% irregular cases are handled by a supervised learning algorithm.

Next a system called illustration2vec is used to create a vector (this is the "embedding" idea though they don't use the word) describing the scene. The output of this is tags like "boy", "girl".

To do the actual translation, the input text is all concatenated together, translated all at once, and then broken back apart into the proper speech bubbles on the other end. To make this work, though, tags from illustration2vec are inserted into the text, which provide context to the translation system.

All of this glosses over additional complex issues that the researchers had to solve to make this work, such as identifying where text is in an image, figure out what style the text is written in, since text in manga is written in various styles, and extract the text.

Towards Fully Automated Manga Translation

#solidstatelife #ai #computervision #nlp #translation #embeddings #manga
 
Female historians and male nurses do not exist, Google Translate tells its European users – AlgorithmWatch

Mostly due to the fact that it uses English as bridge language. It would have been more interesting to compare it with other translating services.

#translation #language #machineLearning
 

Any translation needs?


Hi. I have a friend who is in her last year of the #translation school. She needs to cover certain number of words of English-Spanish translation to graduate. It could be a good opportunity to contribute to a #free #software project. If you need translation help, drop me a comment.
 

Any translation needs?


Hi. I have a friend who is in her last year of the #translation school. She needs to cover certain number of words of English-Spanish translation to graduate. It could be a good opportunity to contribute to a #free #software project. If you need translation help, drop me a comment.
 

The Best Ways to Use Google Translate in a Foreign Country | Lifehacker


'If all you’ve ever done with Google Translate is copy-and-paste text, you’re missing out. The app can translate signs and menus visually, save your most-used phrases, and more.'

TLDR: But don't trust it too far

#technology #translation
 

The Best Ways to Use Google Translate in a Foreign Country | Lifehacker


'If all you’ve ever done with Google Translate is copy-and-paste text, you’re missing out. The app can translate signs and menus visually, save your most-used phrases, and more.'

TLDR: But don't trust it too far

#technology #translation
 
Later posts Earlier posts