add: save to disk

Konnagar.txt

@@ -0,0 +1,66 @@
+Konnagar is a town and municipality of Hooghly district in West Bengal, India. A railway station connects Konnagar with the city of Howrah. The town is under the jurisdiction of the Uttarpara police station in  and the Kolkata Metropolitan Development Authority (KMDA).
+
+ History 
+
+
+In 1931, Tomáš Baťa, the Czechoslovak shoe tycoon, established his first Indian operation in Konnagar. By 1936, the plant was phased out. In May 1931, Baťa sold his business interests to his brother Jan Antonín Baťa, who founded Batanagar, Batas first permanent shoe factory in India. The  brand was launched on 24 August 1894 in Zlín, then in Austria-Hungary (now in the Czech Republic). The company first established itself in India in 1931 by renting a building to start an experimental shoe production plant in Konnagar, with 75 Czech experts. Jan Antonín Baťa also built factories in Digha, near Patna, and elsewhere in India, employing more than 7,000 people. Batanagar, under Baťas ideals, became one of the bigger suburban towns near Kolkata.
+
+Konnagar has been the home of many notable personalities. It was the ancestral home of the great nationalist and religious leader Sri Aurobindo and noted physicist Sisir Kumar Mitra. Shafiur Rahman one of the martyrs of the language movement in East Pakistan was born in Konnagar. 
+
+The town had been visited by the greats of , Mahasweta Devi and many others. Tagore was believed to be just a child when he and his family came to this town to escape the onslaught of Dengue which hit Kolkata. He revisited the town again when he was 19. He came to visit the Brahmasomaj Ghat along with his father . Shibram Chakraborty used to live here for a brief period of time in his childhood near G.T.Road beside Konnagar High School.Sri Aurobindos father and famous Indian geographer Shashi Bhusan Chatterjee happened to be a pupil of Konnagar High School established in the 19th century. And the town though got recognition chiefly due to the initiatives of Sib Chandra Deb, its antiquity can hardly be denied. Its reference is there even in the 500-year-old Mangal-Kāvya.
+
+Konnagar is well known for its Shakuntala Kali Temple. This temple is considered to be very sacred and people from various parts of West Bengal pay a visit. This is a Puja that was started by the Chakraborty Bari Zamindars and today it has become an important event in Konnagars calendar. The annual Puja is held in the Bengali calendar month of Boisakh (April). A very interesting fact about this Puja is that the idol is completed on the day of Puja itself. There are grand Fairs organised on the grounds adjacent to the temple.
+
+Apart from this Puja, there is another important puja in this small city. Rajrajeshwari Puja has been organised in Konnagar for over 300 years now. It is held on the eve of Maghi Purnima (February).
+
+Geography
+Konnagar is located at . It has an average elevation of 14 metres (45 feet). Konnagar is positioned between Rishra and Hind Motor on the Howrah-Bardhaman main line and Grand Trunk Road. It is located on the west bank of the River Hooghly. Its approximate area is 4.32 km2.
+
+West side of the Konnagar railway station is called Nabagram and Kanaipur. Most of the population in Nabagram is migrated from East Bengal. Being situated only 13.5 km from the heart of Kolkata, the area enjoys semi urban status. Konnagar has an extensive fruit and vegetable market which is visited by people from the nearby areas. The market is famous for its affordable prices. Konnagar is filled with many cultural clubs which makes it a culturally rich town.
+
+Konnagar is also known for its famous Shakuntala Kalitala Mandir, Baro Mandir, Rajrajeswari Mandir, Shankaracharya Temple, Bungalow of Abanindranath Tagore, Paternal house of Rishi Aurobindo Ghosh, Bramhosomaj Ghat and house made by Raja Rammohan Roy. 
+The first ever workshop of Bata Shoe Company Ltd. in India was initially set up in Konnagar in 1932. Baro Mandir (Dwadosh Mandir Ghat/Twelve Temples) was built in 1821 in the bank of Ganges river. The Bungalow of Abanindranath Tagore is a heritage place of the city.
+
+Demographics
+
+
+As per 2011 Census of India Konnagar had a total population of 76,172 of which 38,653 (51%) were males and 37,519 (49%) were females. Population below 6 years was 5,815. The total number of literates in Konnagar was 63,911 (90.84% of the population over 6 years).
+
+The following Municipalities and Census Towns in Hooghly district were part of Kolkata Urban Agglomeration in 2011 census: Bansberia (M),  (M), Bara Khejuria (Out Growth), Shankhanagar (CT), Amodghata (CT), Chak Bansberia (CT),  (CT), Kodalia (CT), Kulihanda (CT),  (CT),  (CT),  (M), Champdani (M), Chandannagar (M Corp.), Baidyabati (M), Serampore (M), Rishra (M),  (CT), Bamunari (CT), Dakshin Rajyadharpur (CT), Nabagram Colony (CT), Konnagar (M), Uttarpara Kotrung (M), Raghunathpur (PS-Dankuni) (CT), Kanaipur (CT) and  (CT).
+
+ Transport 
+
+
+Konnagar is well connected with Howrah through trains. Konnagar lies on the Howrah-Bardhaman main line. Konnagar railway station has three platforms (1 for Down train, 2 for both and 3 for Up trains). Konnagar is also connected to Kolkata through roadways by / Grand Trunk Road.
+
+Private bus
+ 2 Chunchura Court - Dakshineswar
+ 285 Serampore - New Town
+3 Serampore - Bagbazar/
+D4/1 Serampore - Bagbazar( up to Belgachia)
+There are regular auto services available also towards  and Serampore. Apart from this, there are Ferry Services to Greater Kolkata (Panihati) on the other end of Ganges banks. Now ferry service becomes more frequent. Netaji Subhas Chandra Bose International Airport is 17 km from Konnagar.
+
+Education
+
+Nabagram Hiralal Paul College, a general degree college, was established in 1957, at Nabagram, Konnagar. Prof. Sudhangshu Sekhar Bhattacharya, from the Bhattacharya family of Nabagram was one of the early member of this college and a big contributor to the build-up of the college. Every year there is a alumni meet on first Sun day of the year. In Konnagar, there are quite a few higher secondary high schools including Konnagar Rajendra Smriti Vidyalaya, Konnagar Hindu Girls High School, Konnagar Boys High School, Nabagram Hiralal Paul Girls High School and Nabagram Vidyapith. There are many government secondary schools including ASHALATA BALIKA VIDYALAYA (HIGH), KONNAGAR K.P. BALIKA S.S., KONNAGARSRI ARABINDA VIDYAPITH, Konnagar Nagendra Nath kundu Vidya Mandir. All the above mentioned institutions are Govt. hold. Apart from it there are various (more than 5) private educational institutions are there of all different Boards.
+
+Notable people 
+ Aurobindo Ghosh (Rishi), a famous freedom fighter.
+ Premendra Mitra a renowned Bengali writer is connected with Konnagar by family.
+ Shafiur Rahman, martyr of the 1952 .
+ Digambar Mitra, famous Kolkata-based descendant of Mitra family of Konnagar.
+ Sisir Kumar Ghosh, Social reformer.
+ Sisir Kumar Mitra, Scientist and pioneer of radio-science in India.
+
+ References 
+
+
+
+
+
+
+
+
+Category:Cities and towns in Hooghly district
+Category:Neighbourhoods in Kolkata
+Category:Kolkata Metropolitan Area

readme.md

@@ -6,4 +6,5 @@ Todo:
 - create separate text files for each  link
 - output in a directory
 - progress bar
-- helpful errors
+- helpful errors
+- fix {} escape

src/main.rs

@@ -1,5 +1,10 @@
-use clap::Parser;
-use std::path::PathBuf;
+use clap::{builder::PossibleValue, Parser};
+use std::{
+    fs::File,
+    io::Write,
+    os::linux::raw::stat,
+    path::{Path, PathBuf},
+};
 use url::Url;
 
 use utils::*;
@@ -10,8 +15,11 @@ mod utils;
 #[command(about = "Wikipedia on your terminal.")]
 #[command(version, long_about = None)]
 struct Args {
-    #[arg(short, long)]
+    #[arg(short, long, help = "link to the wikipedia article")]
     link: String,
+
+    #[arg(short, long, help = "save to disk", action)]
+    save: bool,
 }
 
 #[derive(Debug)]
@@ -62,27 +70,30 @@ fn main() {
     let url = Url::parse(&link).unwrap();
     let wikipedia_url = url.host_str().unwrap();
 
-    let (article_title, raw_text) = get_article(format!("https://{wikipedia_url}/w/api.php?action=query&format=json&prop=revisions&titles={url_title}&formatversion=2&rvprop=content&rvslots=*")).unwrap();
+    let raw_text = get_article(format!("https://{wikipedia_url}/w/api.php?action=query&format=json&prop=revisions&titles={url_title}&formatversion=2&rvprop=content&rvslots=*")).unwrap();
     // Trimming the fat
     let slice_index = raw_text.find("\"").unwrap();
-    let mut characters: Vec<char> = raw_text[slice_index..].chars().collect();
+    let mut characters: Vec<char> = raw_text[slice_index + 1..].chars().collect();
     characters.pop();
     characters.push('\0');
     let tokens = parse_text(&characters).unwrap();
-    for token in tokens {
-        token.print(&characters);
+    let plaintext = generate_plaintext(&tokens, &characters);
+    if args.save {
+        save_to_disk(&plaintext, &(url_title.to_string() + ".txt"));
+    } else {
+        output_to_stdout(&plaintext);
     }
 }
 
-fn get_article(url: String) -> Result<(String, String), String> {
+fn get_article(url: String) -> Result<(String), String> {
     let response: serde_json::Value = reqwest::blocking::get(url)
         .map_err(|err| format!("Error: Could not fetch article due to {}", err))?
         .json()
         .map_err(|err| format!("Error: JSON conversion failed due to {}", err))?;
-    Ok((
-        response["query"]["pages"][0]["title"].to_string(),
+    Ok(
+        // response["query"]["pages"][0]["title"].to_string(),
         response["query"]["pages"][0]["revisions"][0]["slots"]["main"]["content"].to_string(),
-    ))
+    )
 }
 
 // little utils : move to a separate file
@@ -239,3 +250,49 @@ fn parse_text(characters: &Vec<char>) -> Result<Vec<Token>, String> {
 
     Ok(tokens)
 }
+
+fn generate_plaintext(tokens: &Vec<Token>, characters: &Vec<char>) -> String {
+    let mut plaintext = String::new();
+    let get_text = |token: &Token| {
+        characters[token.start..token.start + token.length]
+            .iter()
+            .collect::<String>()
+    };
+    for token in tokens {
+        match token.format {
+            FormatType::Title
+            | FormatType::Bold
+            | FormatType::Italic
+            | FormatType::PlainWord
+            | FormatType::Subtitle
+            | FormatType::Subsubtitle
+            | FormatType::WikiLink
+            | FormatType::BulletBold
+            | FormatType::BulletItalic
+            | FormatType::InlineQuote => plaintext.push_str(&get_text(token)),
+
+            FormatType::Space => plaintext.push(' '),
+            FormatType::NewLine => plaintext.push('\n'),
+        }
+    }
+    plaintext.trim().to_string()
+}
+
+fn output_to_stdout(plaintext_string: &str) {
+    println!("{}", plaintext_string);
+}
+
+fn save_to_disk(plaintext_string: &str, article_title: &str) {
+    let path = Path::new(article_title);
+
+    let mut file = match File::create(&path) {
+        Err(why) => panic!("Error: Couldn't create {}: {}", path.display(), why),
+        Ok(file) => file,
+    };
+
+    // Write the `LOREM_IPSUM` string to `file`, returns `io::Result<()>`
+    match file.write_all(plaintext_string.as_bytes()) {
+        Err(why) => panic!("Error: Couldn't write to {}: {}", path.display(), why),
+        Ok(_) => println!("Saved to {}", path.display()),
+    }
+}