readability/README.md

95 lines
2.7 KiB
Markdown
Raw Normal View History

2016-04-15 11:51:29 +00:00
# Readability
2016-04-24 07:14:31 +00:00
[![Build Status](https://travis-ci.org/keepcosmos/readability.svg?branch=master)](https://travis-ci.org/keepcosmos/readability)
[![Readability version](https://img.shields.io/hexpm/v/readability.svg)](https://hex.pm/packages/readability)
2016-04-25 01:00:04 +00:00
Readability is a tool for extracting and curating the primary readable content of a webpage.
2016-04-24 07:14:31 +00:00
Check out The [Documentation](https://hexdocs.pm/readability/Readability.html) for full and detailed guides
2016-04-15 11:51:29 +00:00
## Installation
If [available in Hex](https://hex.pm/docs/publish), the package can be installed as:
1. Add readability to your list of dependencies in `mix.exs`:
2016-04-24 07:14:31 +00:00
```elixir
def deps do
[{:readability, "~> 0.3"}]
end
```
2016-04-15 11:51:29 +00:00
2. Ensure readability is started before your application:
2016-04-24 07:14:31 +00:00
```elixir
def application do
[applications: [:readability]]
end
```
## Usage
To parse document, you must prepare html string.
2016-04-24 09:40:35 +00:00
The example below, `html` variable is the html source from [Elixir Design Goals](http://elixir-lang.org/blog/2013/08/08/elixir-design-goals/)
2016-04-24 07:14:31 +00:00
### Examples
```elixir
### Extract the title
Readability.title(html)
#=> Elixir Design Goals
### Extract the content with transformed html.
2016-04-24 09:40:35 +00:00
html
|> Readability.article
|> Readability.raw_html
2016-04-24 07:14:31 +00:00
#=>
# <div><div class=\"entry-content\"><p>During the last year,
# ...
2016-04-24 09:40:35 +00:00
# ... out our sidebar for other learning resources.</p></div></div>
2016-04-24 07:14:31 +00:00
### Extract the text only content.
2016-04-24 09:40:35 +00:00
html
|> Readability.article
|> Readability.readable_text
2016-04-24 07:14:31 +00:00
#=>
# During the last year, we have spoken at many conferences spreading the word about Elixir. We usually s.....
# ...
2016-04-24 09:40:35 +00:00
# ... started guide, or check out our sidebar for other learning resources.
2016-04-24 07:14:31 +00:00
```
### Options
2016-04-24 09:40:35 +00:00
You may provide options(Keyword type) to `Readability.article`, including:
2016-04-24 07:14:31 +00:00
2016-04-24 09:40:35 +00:00
* retry_length \\\\ 250
* min_text_length \\\\ 25
* remove_unlikely_candidates \\\\ true,
* weight_classes \\\\ true,
* clean_conditionally \\\\ true,
* remove_empty_nodes \\\\ true,
2016-04-24 07:14:31 +00:00
## Test
To run the test suite:
$ mix test
2016-04-24 09:40:35 +00:00
## Todo
* [ ] Extract authors
2016-04-24 07:14:31 +00:00
* [ ] Extract Images
2016-04-24 09:40:35 +00:00
* [ ] Extract Videos
2016-04-24 07:14:31 +00:00
* [ ] Convert relative paths into absolute paths of `img#src` and `a#href`
* [ ] More configurable
* [ ] Command line interface
## Related and Inpired Projects
* [readability.js](https://github.com/mozilla/readability) is a standalone version of the readability library used for Firefox Reader View.
* [newspaper](https://github.com/codelucas/newspaper) is an advanced news extraction, article extraction, and content curation library for Python.
* [ruby-readability](https://github.com/cantino/ruby-readability) is a tool for extracting the primary readable content of a webpage.
## LICENSE
This code is under the Apache License 2.0. See <http://www.apache.org/licenses/LICENSE-2.0>.