Searching Daily Kos
Quick Search Tips
Search: enter words you want to find in the target collection. By default only targets containing all the words will be returned. Capitalization is ignored. If you want to find several words in a precise order (a phrase) put double quotes around them as in "this is a phrase". Single quotes are not permitted, and are removed. Although "AND" is assumed by default, "OR" and "NOT" are supported, as follows.
Multiple query words are by default understood to be joined by "AND". Certain characters have to be "escaped" by placing a back slash in front of them; to find "/", use "\/", to find "=", use "\=". Complex queries can be constructed using parentheses and restricted queries. Restricted queries: limiting the parts of the document to be searched. The available options are described here. For both stories and comments (documents):
For stories only:
For comments only:
Find: In order to choose what kind of documents to find, use the pull down menu to select one of the following. Each option is identified as either a database search, meaning that the primary database is searched using Scoop directly, or index search, meaning that a specially constructed index of words contained in documents is searched using swish-e. There are three different indices, one for diaries (aka user diaries), one for stories (aka front page stories), and one for comments. User diaries that are elevated to the front page are found in both story and diary indices.
From: and To: pull down menus restrict the time interval of the search. The default interval is from one day ago to most recent. Most recent may be as much as 5 minutes out of date. Sort by: The order of presentation of results. Not all choices are relevant to all searches. Story and Diary Search sort choices.
Comment search sort choices shared with stories and diaries are "Relevance", "Title", and "Time", which are as described above. Those specific to comment are "Cid", "Nrec", and "Ntroll".
More example queries
For comments only Everything past the "#" is a comment, not part of the query!
Searching Daily Kos with election tags (etags)You can now search using election tags as follows. etag=CT-SEN Etags with embedded spaces should be quoted. etag="CA-LT. GOV" You can only use one etag per search. Good election tags have a two letter postal state abbreviation hyphen seperated from either a house district number, or a statewide office abbreviation (SEN, GOV, Lt.GOV, SOS, etc). There is one exception, etag="2006 ELECTION" also works. Good etags are ones we already know about by finding them in the tag list of an existing diary or by appearing in one of the election roundup diaries. The etag is used to build a more complicated query that looks for the tag in tags, title, or body text (stories/diaries) or in subject and comment text (comments). Certain meta dairies are excluded, including open threads, jotter's daily lists, and sidinny's election roundup, and the top comments diaries.
Software used.The search facility currently installed is based on the swish-e search engine: it is fast (compiled C code), open source, and flexible (you can feed it documents out of a database using your own custom preproccessor). It is intended for document collections numbering a million or less, which will do for Daily Kos with under half a million documents (excluding comments). In addition, swish-e offers many of features most desired, including search of phrases and search within configurable document areas (title, author, tags, URLs). What is indexed. All Daily Kos documents from the beginning to within 5 minutes of the present moment are indexed. Older documents are stored in static yearly indices while more recent documents are indexed and reindexed, frequently at first, then less frequently as they age, until they roll over into a growing, static, permenant yearly index. The right mix of number of index files and frequency of update to obtain the optimal combination of ease of maintenance and speed of search may require further experimentation. At present three different series of indices are maintained, corresponding to (in order of increasing total number of documents) front page stories, user diaries, and comments. For 2005 the number of stories, diaries, and comments was 5097, 88943, and 693224, respectively, containing 56993, 371578, and 2945916 distinct words. Design Considerations. The utility of search to a community of users is delivered not simply by the results of the text search, but most critically by the ordering of the results such that highest "value" hits are displayed first. (Most search researchers know that no one looks beyond about the first handful of results; almost no one looks beyond the first page.) The key to arranging for best to come first is to understand, have access to, and use the meta data associated with the text documents searched. Meta Data. At Daily Kos, we have a number of sorts of meta data that are relevant to documents. First among these is the time of posting. What is the latest? When did they say that? Who was first? That sort of thing. So the one thing search has to get right is time, in the sense of being up to date, that is including the latest diaries, as well as in the sense of permitting searches to be organized by reference to the present time. Making it up to date. Because recommendations are only open for 24 hours after a diary is published, it is important that search indices be refreshed sufficiently often during that interval to provide reasonably up to date information on total recommendations and comments. Currently, diaries up to 6 hours old are refreshed every 5 minutes, and diaries 6 to 24 hours old are refreshed every 15 minutes. More Meta Data. Other important meta data associated with stories and diaries includes comments and recommendations. The number of recommendations offers insight, imperfect, but useful, into the degree of approval a diary has received from the Daily Kos community. The number of comments indicates the size of the discussion engendered. The two numbers together give an interesting insight into a diaries history at Daily Kos. Some user diaries are promoted to the front page, and some diarists ("front pagers") are allowed to post directly to the front page. The latter are not open for recommendation. In order to harmonize listings from searches which include both front page stories and user diaries, a plausible value for number of recommendations (83) is assigned to front page stories, but in lists, this number is indicated by "*" to avoid placing too much emphasis on this made up number. The search query. Search queries can contain individual words, all of which must be found in a text document to constitute a hit. Queries can also contain phrases, indicated by double quotes around the phrase. All words are converted to lower case while searching, so letter case is of no consequence. In addition, queries can include boolean terms, such as "and", "or", and "not" which can be used to construct more complex and perhaps more informative results. By default "and" is assumed between individual words in a query. Finally, queries or parts of queries can be restricted to only search within parts of a document or document metadata. The parts available are shown above, along with examples. Certain characters have special meaning to the search engine (open and close paren, double quote, equals sign). Only a restricted list of characters are expected to occur in a word. For the indices here at daily kos, the word characters are as follows.
All other characters (non word, non special) are silently replaced by spaces while searching. For instance searching with "Jerome a Paris", "Jerome.a.Paris", or "Jerome,a,Paris" all give the same result. This is important in certain cases, for example searching for the url http://riverbendblogspot.com is accomplished with the query site=("riverbend.blogspot.com"). The reasonable looking but incorrect query site=riverbendblog.blogspot.com doesn't work as expected because in the absence of the double quotes and parentheses, replacement of "." with " " and the default boolean AND understood between search words makes it equivalent to site=riverbendblog AND blogspot AND com, which returns as hits only documents with riverbendblog in a url, and blogspot and com in the body text. More on Search
If you are intriqued by search and would
like to know more about the basics of how it is done, this is a
good introduction.
On Search. Basic basics. by Tim Bray. Since I've emphasized the importance of meta data, I'll also point to another article from the same series, On Search. SearchMeta, with some further thoughts on things meta. While I'm on the subject of underlying technologies, let me end this section by pointing to an article by Tim Bray on Ajax, the other new addition here at Daily Kos. In it, he also gives credit to Jesse James Garrett for the Ajax acronym in the process of thoroughly describing it.
Daily Kos is an amazing place, community, phenomenon. It has been my
privilege to have the opportunity to contribute.
-- jotter |
Recommended
- CNN has published a deeply troubling report on the torture that Israel is committing on Gazan detainees at the Sde Teiman prison in Israel’s Negev desert. The Israeli whistleblower revealed that detainees, some of whom are innocent surgeons and...166 comments 254 Recs
- I know that the GOP believes a lot of ridiculous nonsense. It’s their stock-in-trade. �But hearing this nutty theory espoused by the current Speaker of the House is more than a little jarring.44 comments 73 Recs
- UPDATE: Friday, May 10, 2024 · 2:15:25 PM +00:00 · distraught Over 2 months ago, Biden nominated someone to fill 1 of 2 vacancies on the Postal Board. Walsh would be the fourth Democrat to sit on the board, joining three Republicans and one...143 comments 207 Recs
- Good morning, trial chickadees. As always, my comments and those of DRo will be in the comments not updates. We both appreciate the great input from the community at large. I will add a link to Mark Sumner’s diary at the top of this intro when he...392 comments 207 Recs
- Primum non nocere (first, or above all, do no harm) is a Latin expression generally applied to the medical profession. However, I viewed it as equally applicable to the practice of law and often reminded myself of it in my own practice. Trump’s...383 comments 872 Recs
- A Severe G4 Geomagnetic Storm is expected to lash at earth around 03:00 — 09:00 UTC Saturday morning (11 p.m. EDT Friday night to 5 a.m. EDT Saturday morning). A series of strong solar flares and Coronal Mass Ejections took place Wednesday and...27 comments 159 Recs
- A lot has changed since I first wrote about E-bikes back in 2020. Today sales of E-bikes are vastly outpacing sales of electric cars. Seems like they’re everywhere now. In 2023 roughly 40 million E-bikes were sold worldwide. I was just at the local...74 comments 157 Recs
- So, he could be very nervous then I guess. Maybe a sign that the walls are closing in for Trump at his current trials so Mr. Navarro is very busy though in jail. www.rawstory.com/… "Former Trump advisor Peter Navarro files emergency motion to get his...33 comments 62 Recs
- Welcome 🌻 to Fridayâs Roundup of Good News! I admit, sometimes the news scares me. Thereâs a lot of evil in the world. When I look at Hamas, when I look at Russia, when I look at Iran and the ...112 comments 148 Recs
- Sometimes I am disheartened when otherwise good and rational people pile on to make fun of other people’s sojourns through life and the medical crises which inherently evolve from being alive on this planet. I am no fan of RFK Jr as a candidate, but I...53 comments 38 Recs
- The current intensification of Russian attacks may be a result of the fact that the soil has dried out after the spring mud season, which facilitates more rapid mechanized maneuver, and that Russian forces are trying to take advantage of Ukraine's...26 comments 143 Recs
- I've been reading comments in Times of Israel strings attached to articles about Biden's decision to temporarily withhold 2,000 pound bombs, and other higher intensity unguided offensive weaponry, to reduce the chance that Netanyahu will to do to Rafah...86 comments 128 Recs
Recent Diaries
- Hi Peeps! There doesn’t appear to be a Friday diary in the queue, so I’m putting up a rerun. I will delete it if another diary shows up. The Toosdai Critters have a play date. Play dates may include: play bows butt wiggles stealthiness wrassling...4 comments 4 Recs
- Being a Garbahz (pronounced like “garage”) Cook means throwing meals together without using a recipe. Ingredients often come from what I have on hand as leftovers (also known as “Refrigerator Cooking”), or what I got on sale or even for free....1 comments 0 Recs
- Listen to our archived episodes: Pandora|LibSyn|YouTube Support the show: Patreon|PayPal: 1x or monthly|Square Cash * David Waldman makes a surprise return to KITM! It was mostly a surprise to him, but there he was, sitting in front of the microphone,...1 comments 0 Recs
- From BBC News: “A Virginia school board has approved a motion for two schools to revert to their Confederate names following a debate that bitterly divided a town. The Shenandoah County School Board voted 5-1 to reinstate the names of Stonewall Jackson...3 comments 8 Recs
- Donald Trump is just now beginning to learn something rather important. Most people learn things such as this in their very young childhood. Mr. Trump is in his seventies, but it can fairly be said that he is also in some respects still in his very...1 comments 1 Recs
- This cartoon was written by me and drawn by the wonderful cartoonist Jenn Manley Lee. (The link goes to Jenn's long-running science fiction slice-of-life comic, "Dicebox," which I highly recommend). Transcript, comments, and a blog post about this...2 comments 6 Recs
- When Louis IVX declared “I am the State” he created the template for all autocrats to follow. Today’s world is being distorted by deluded individuals who believe they are ordained by fate to rule over everyone else. Trump, Putin, Netanyahu, Orban, Kim...1 comments 6 Recs
- Once again Barak Ravid of Axios is the first to report that “last night the Israeli War Cabinet voted to expand its area of Rafah operations.” This can be construed as a violation of President Biden’s red line which he articulated to CNN recently. The...3 comments 4 Recs
- Social media continually elevates both genuine acts of antisemitism hijacking this movement and innocuous mistakes such as this one. This video has been reported by many people over the days since the magazine first shared its coverage with the...2 comments 0 Recs
- Early this morning a Russian oil refinery in Kaluga Oblast had a full 30 minutes warning that a Ukrainian drone was headed their way. The drone arrived right on time to hit the facility …. unmolested and with devastating effect. What air defense doing?...5 comments 17 Recs
- This is a very short and simple diary. Any even moderately educated person knows that any defendant has the right to testify in their own defense if they choose to do so. , If a person paid attention in high school, but lacked the opportunity to go...14 comments 10 Recs
- I know that the GOP believes a lot of ridiculous nonsense. It’s their stock-in-trade. �But hearing this nutty theory espoused by the current Speaker of the House is more than a little jarring.44 comments 73 Recs
- I make no bones about the fact that i like the last state canvassing diary of the week to be about the state that turns out the most volunteers the prior Saturday. I like to honor the Hope Springs from Field PAC [website] volunteers from that state and...3 comments 11 Recs
- Mug cakes are single serving cake (well, some recipes make two servings) that you mix and bake in a large mug by microwaving it. �It’s hard to figure out when mug cakes became a thing but there ...1 comments 2 Recs
- Over the past 3 election cycles, I’ve raised over $7.7 MILLION for HUNDREDS of Democratic candidates at the federal, statewide and state legislative levels. For statewide races, I list every GOP state, no matter how deep red, because I trust people to...1 comments 10 Recs
- Sometimes I am disheartened when otherwise good and rational people pile on to make fun of other people’s sojourns through life and the medical crises which inherently evolve from being alive on this planet. I am no fan of RFK Jr as a candidate, but I...53 comments 38 Recs
- Full disclosure: I am a low level administrator at Middlebury College. My wife, however, is a big wig here and was directly involved in this resolution. Can’t help it, but I feel like this is a BFD: In “How My Campus Avoided a Horrible Protest...4 comments 5 Recs
- So, he could be very nervous then I guess. Maybe a sign that the walls are closing in for Trump at his current trials so Mr. Navarro is very busy though in jail. www.rawstory.com/… "Former Trump advisor Peter Navarro files emergency motion to get his...33 comments 62 Recs
- “Watch out! Be on your guard against all kinds of greed…” – Jesus (Luke 12:15) “My whole life I’ve been greedy, greedy, greedy. I’ve grabbed all the money I could get. I’m so greedy.” – Donald Trump The Washington Post revealed yesterday that Donald...3 comments 19 Recs
- Steve Bannon keeps his main narrative what is mainly his innocence. The spectator is still not knowing why he didn’t appear in front of the committee then. The judges remain pretty much unconvinced as well www.rawstory.com/… Steve Bannon moved one step...31 comments 78 Recs