python - RegEx Tokenizer to split a text into words, digits and punctuation marks -

-

what want split text ultimate elements.

for example:

from nltk.tokenize import * txt = "a sample sentences digits 2.119,99 or 2,99 awesome." regexp_tokenize(txt, pattern='(?:(?!\d)\w)+|\s+') ['a','sample','sentences','with','digits','like','2.199,99','or','2,99','are','awesome','.']

you can see works fine. problem is: happens if digit @ end of text? 

txt = "today it's 07.may 2011. or 2.999." regexp_tokenize(txt, pattern='(?:(?!\d)\w)+|\s+')  ['today', 'it', "'s", '07.may', '2011.', 'or', '2.999.']

the result should be: ['today', 'it', "'s", '07.may', '2011','.', 'or', '2.999','.']

what have result above?

i created pattern try include periods , commas occurring inside words, numbers. hope helps:

txt = "today it's 07.may 2011. or 2.999." regexp_tokenize(txt, pattern=r'\w+([.,]\w+)*|\s+') ['today', 'it', "'s", '07.may', '2011', '.', 'or', '2.999', '.']

Comments

Post a Comment

Popular posts from this blog

c# - How to set Z index when using WPF DrawingContext? -

-
how set z-index drawing object when using drawingcontext.drawxxx() methods? the object drawn last have higher z index. can't change index of drawn objects. way draw in order. if using wpf (as placed tag), can use, example, canvas control. create shapes need like polyline obj = new polyline(); //... // ... set properties of obj and add them canvas uielementcollection: yourcanvasname.children.add(obj); //or yourcanvasname.children.insert(i, obj); first items of collection have higher z index. advantages in way: no need redraw on window changes, can anytime move objects , change order.
Read more

razor - Is this a bug in WebMatrix PageData? -

-
i think may have found bug in webmatrix's pagedata, not sure. concerns how pass data partial page calling page. in webmatrix documentation (tutorials, e.g. " 3 - creating consistent look ", , example code), pagedata recommended mechanism pass data between pages (e.g. content page layout page, or partial page). however have found not work other way, pass data partial page calling page. modifying or adding entries in pagedata in partial page, not seem calling page. cutting right down simplest possible example, in test page may have this: @{ pagedata["test"] = "initial entry"; } <p>before calling partial page, test value @pagedata["test"]</p> @renderpage("_testpartial.cshtml") <p>after returning calling page, test value @pagedata["test"]</p> and in _testpartial.cshtml page might have this: @{ pagedata["test"] = "modified entry"; } <p>in partial page...
Read more

android - layout with fragment and framelayout replaced by another fragment and framelayout -

-
edit: so after comments below, revisted , realized hanging me up. imagine client list , client details activity started : public class clientsmainactivity extends fragmentactivity { @override public void oncreate(bundle savedinstancestate) { super.oncreate(savedinstancestate); //studiotabopenhelper db; setcontentview(r.layout.main_client_activity); } } so works great, starts main_client_activity (defined in layout below, , call activity when button on main screen clicked): intent intent = new intent(getactivity(), clientsmainactivity.class); startactivity(intent); easy issue is, clientsmainactivity not call oncreateview or anything, sets layout layout defines fragment, , listfragment. fine cause not trying pass clientsmainactivity , if have hypothetical activity like: sessionmainsactivity called when click on session edit of client, not calling sessionsmainactivity same way (starts activity sets alayout), want layout set defines...
Read more