I have an array of words and and I would like to remove all words that contain any unusual characters like umlauts, accents, etc. (I know there are ways to normalize them to regular characters instead, but I specifically want to remove them). My idea so far is to create an array of accepted characters (letters a-z), because this is easier than making a blacklist accounting for all possible combinations of letters and accents, and go through my array of words checking if the word has any characters other than the accepted ones. I've found  article that describes the opposite, removing all words that do contain a certain characters: filtered_tokens = [w for w in tokens if all(ch not in w for ch in accepted_characters)] unfortunately a simple negation doesn't make this work for my problem. I'm open to suggestions and new approaches altogether, but ideally I would like to get this to work with no additional packages besides maybe nltk, which I am using to extract my words from a text.
122327
I want to I check whether a string is in ASCII or not.  I am aware of ord(), however when I try ord('é'), I have TypeError: ord() expected a character, but string of length 2 found. I understood it is caused by the way I built Python (as explained in ).   Is there another way to check?
49924
I put together a script to do some file operations for me. I am using the wild card operator * to apply functions to all files of a type, but there is one thing I don't get. I can unzip all files in a folder like this  unzip "*".zip   However, to remove all zip files afterward, I need to do   rm *.zip   That is, it does not want the quotation marks. The unzip, on the other hand, does not work if I just give it the * (gives me a warning that "files were not matched").   Why is this different? To me, this seems like the exact same operation. Or am I using the wild card incorrectly?   in Unix do not really go into this, and I could not locate anything in the rm or zip docs.  I am using the terminal on a Mac (Yosemite).
103966
Can you create an abbreviation for any word that has no commonly accepted abbreviation simply by dropping some or all the vowels (i.e., restore:  rstr or restr; replace:  rplc or replc)?
114049
I just read the answers to  and everyone said to delete the last character. My question is,  isn't the eof character the last one?
19426
If I have a string "1 2 3 2 1" - or an array [1,2,3,2,1] - how can I select the unique values, i.e.  "1 2 3 2 1" produces "1 2 3"    or  [1,2,3,2,1] produces [1,2,3]   Similar to uniq but uniq seems to work on whole lines, not patterns within a line...
116832
How can I filter entries containing any specific word in a list of words? For example, I have a list of road names in Singapore.  Amoy Street, Singapore Ann Siang Hill Anson Road Arab Street Armenian Street, Singapore BBaghdad Street (Singapore) Balestier Road Banda Street Bartley Road Beach Road, Singapore Bencoolen Street Bernam Street Boat Quay Boon Tat Street Boundary Road, Singapore Bras Basah Road Bugis Street Bukit Batok Road Bukit Pasoh Road Bukit Timah Road CCantonment Road, Singapore Choa Chu Kang Road Clarke Quay Clementi Road Club Street Collyer Quay Connaught Drive Craig Road (Singapore) Cross Street  and many more  My spreadsheet has a large number of entries like the following, which may or may not contain road names mentioned in my list:   Saw an accident at Thomson Road Found this by accident 6 vehicles crashed at Balestier Road I wanna crash now. So tired. Bus collides with bicycle at Arab Street. Accident at City Road. You can crash my house later.   How do I filter to return entries that contains any road name identified in the list of names? How do I introduce an array/list of road names into Microsoft Excel and then relate it to a filter function?
57802
I have a regex created from a list in a database to match names for types of buildings in a game. The problem is typos, sometimes those writing instructions for their team in the game will misspell a building name and obviously the regex will then not pick it up (i.e. spelling "University" and "Unversity").  Are there any suggestions on making a regex match misspellings of 1 or 2 letters?  The regex is dynamically generated and run on a local machine that's able to handle a lot more load so I have as a last resort to algorithmically create versions of each word with a letter missing and then another with letters added in.  I'm using PHP but I'd hope that any solution to this issue would not be PHP specific.
41896
I need to compare 2 strings as equal such as these:     Lubeck == Lübeck   In JavaScript.  Why? Well, I have an auto-completion field that's going out to a Java service using Lucene, where place names are stored naturally (as Lübeck), but also indexed as normalized text,   import sun.text.Normalizer; oDoc.setNameLC = Normalizer.normalize(oLocName, Normalizer.DECOMP, 0)     .toLowerCase().replaceAll("[^\\p{ASCII}]","");   This way some-one who doesn't know to type "Mèxico" can type "mexico" and get a match which returns "Mèxico" (among a lot of other possible hits, like "Café Mèxico, Dubai, UAE").  Now the thing is I don't have the ability to change the service to do any highlighting on the server side, therefore I am highlighting on the client JavaScript side with something like:  return result.replace( input.replace(/[aeiou]/g,"."), "&lt;b&gt;$1&lt;/b&gt;");   It's a little more fancy because I am escaping special regex characters in the input. This is fine for simple one word matches at the beginning of a hit, but it really breaks down if you suddenly wish to support multi-word matches like "london cafe":  input = input.strip().toLowerCase(); //fyi prototype's strip is like trim re = new RegEx(input.replace(/[aeiou]/g,".").replace(/\s+/g,"|"),"gi"); return result.replace(re, "&lt;b&gt;$1&lt;/b&gt;");   This doesn't work for say "london ca" (was typing london cafe), because it would mark "Jack London Cabin, Dawson City, Canada" as: "Ja&lt;b&gt;ck&lt;/b&gt; &lt;b&gt;London&lt;/b&gt; &lt;b&gt;ca&lt;/b&gt;bin, Dawson &lt;b&gt;Ci&lt;/b&gt;ty, &lt;b&gt;Ca&lt;b/&gt;nada" [note the "ck" and "Ci" particularly]  Therefore I'm sort of looking for something that's not as crazy as:  input = input.strip().toLowerCase(); input = input.replace(/a/g,"[ÀàÁáÂâÃãÄäÅåÆæĀāĂăĄą]"); input = input.replace(/e/g,"[ÈèÉéÊêËëĒēĔĕĖėĘęĚě]"); // ditto for i, o, u, y, c, n, maybe also d, g, h, j, k, l, r, s, t, w, z  re = new RegEx(input.replace(/\s+/g,"|"),"gi"); return result.replace(re, "&lt;b&gt;$1&lt;/b&gt;");   Is there a compiled table I can refer to mapping a range of characters which are accented versions of an other character to that character, by which I don't mean the plain unicode chart. And if so, could I avoid using weird, possibly slow, RegEx statements?  About the bounty: Before I started a bounty there were two answers, the one pointing me to doing it in Ruby, and  that  wrote which was a completion of the basic form I'd put in my question. Now don't get me wrong, I really appreciate working it out as completely as he did, but I just wished that there might be another way.  It seems so far that everyone who's dropped by to look at the question and answer has decided that MizzardX covers it just fine, or that they have no different approach. I would be interested in a different approach, and if it simply isn't available before the bounty closes, MizzardX will win the bounty (though in a cruel twist, his edits mad it a community wiki answer, so I'm not sure if he'll get the bounty!)
99329
I saw this °͌͌͌͌͌͌͌͌͌͌͌͌͌͌͌͌͌͌͌͌͌͌͌͌͌ strange character online. I also noted that:   It needs 26 backspaces to delete but behaves like one character when selecting.   It is drawn vertically covering many rows above.  What is this character and why is it behaving like this ?
44716
I have a shapefile with names for Regions. These don't seem to contain special characters "Ä", "Ü", "Ö", "ß" but ugly hieroglyphs.  What can I do to display them correctly. If I import the shape as UTF-8 it doesn't work.  The names are in a dbase table. So is it possible to save a dbase table in UTF-8 or is there a different way of dealing with it?
33708
I have a bunch of files on a Ubuntu box, which have various characters in their filenames that Windows doesn't accept (mostly ":" and "*", but possibly others).  What's the simplest way to get these all renamed and moved to a Windows machine? It's OK to replace these characters with something like "[colon]" and "[asterisk]".
104938
Trying to use grep today, I ran into the  of the  in a Unicode file (UTF-8, in this case). Specifically, I was trying to find a file beginning with XYZ with the pattern grep '^XYZ', but of course grep treated the BOM as three separate characters and did not match the first line of the file if the first line started with XYZ. I even tried to update the regular expression to ignore spaces ('^[[:space:]]*XYZ'), but to no avail.  Other questions have dealt with  or , but I want to know if POSIX tools have a general option to handle Unicode files correctly. If grep handled the Unicode file correctly, it would consider the file contents to start after the BOM and match XYZ on the first line just like any other line.
58396
I have a large dictionary file with 300,000+ words in it and I'm trying to find all words with the vowels aeiou in that order and have only exactly 5 vowels. My current attempt does not seem to be working and for the life of me I don't understand why.  less mywords | grep -iE [^aeiou]*a[^aeiou]*e[^aeiou]*i[^aeiou]*o[^aeiou]*u[^aeiou]*   I think it get's all the words I'm looking for but there are a few words that pop up which I can not get rid of.  abstemiousnesses ultraserious   There are a few other but they are in the same vein. Curiously, even if I add something like [^u]* to the front ultraserious keeping popping back up! Any solution would be fine however I would like it restricted to grep as we've been told it can be done using only grep.
26226
I need a robust and simple way to remove illegal path and file characters from a simple string. I've used the below code but it doesn't seem to do anything, what am I missing?  using System; using System.IO;  namespace ConsoleApplication1 {     class Program     {         static void Main(string[] args)         {             string illegal = "\"M&lt;&gt;\"\\a/ry/ h**ad:&gt;&gt; a\\/:*?\"&lt;&gt;| li*tt|le|| la\"mb.?";              illegal = illegal.Trim(Path.GetInvalidFileNameChars());             illegal = illegal.Trim(Path.GetInvalidPathChars());              Console.WriteLine(illegal);             Console.ReadLine();         }     } }
69562
I've been retagging posts matching [special] -character -[character] is:question  The sidebar gives this helpful description of my query:     results found not containing   and not containing   tagged with       not       search options   questions only   It looks like there are some words missing there...
37193
What is the preferred way to remove spaces from a string in C++? I could loop through all the characters and build a new string, but is there a better way?
92412
I have an array of 1000 or so entries, with examples below:  wickedweather liquidweather driveourtrucks gocompact slimprojector   I would like to be able to split these into their respective words, as:  wicked weather liquid weather drive our trucks go compact slim projector   I was hoping a regular expression my do the trick.  But, since there is no boundary to stop on, nor is there any sort of capitalization that I could possibly key on, I am thinking, that some sort of reference to a dictionary might be necessary?    I suppose it could be done by hand, but why - when it can be done with code! =)  But this has stumped me.  Any ideas?