Searching Daily Kos
Quick Search Tips
Search: enter words you want to find in the target collection. By default only targets containing all the words will be returned. Capitalization is ignored. If you want to find several words in a precise order (a phrase) put double quotes around them as in "this is a phrase". Single quotes are not permitted, and are removed. Although "AND" is assumed by default, "OR" and "NOT" are supported, as follows.
Multiple query words are by default understood to be joined by "AND". Certain characters have to be "escaped" by placing a back slash in front of them; to find "/", use "\/", to find "=", use "\=". Complex queries can be constructed using parentheses and restricted queries. Restricted queries: limiting the parts of the document to be searched. The available options are described here. For both stories and comments (documents):
For stories only:
For comments only:
Find: In order to choose what kind of documents to find, use the pull down menu to select one of the following. Each option is identified as either a database search, meaning that the primary database is searched using Scoop directly, or index search, meaning that a specially constructed index of words contained in documents is searched using swish-e. There are three different indices, one for diaries (aka user diaries), one for stories (aka front page stories), and one for comments. User diaries that are elevated to the front page are found in both story and diary indices.
From: and To: pull down menus restrict the time interval of the search. The default interval is from one day ago to most recent. Most recent may be as much as 5 minutes out of date. Sort by: The order of presentation of results. Not all choices are relevant to all searches. Story and Diary Search sort choices.
Comment search sort choices shared with stories and diaries are "Relevance", "Title", and "Time", which are as described above. Those specific to comment are "Cid", "Nrec", and "Ntroll".
More example queries
For comments only Everything past the "#" is a comment, not part of the query!
Searching Daily Kos with election tags (etags)You can now search using election tags as follows. etag=CT-SEN Etags with embedded spaces should be quoted. etag="CA-LT. GOV" You can only use one etag per search. Good election tags have a two letter postal state abbreviation hyphen seperated from either a house district number, or a statewide office abbreviation (SEN, GOV, Lt.GOV, SOS, etc). There is one exception, etag="2006 ELECTION" also works. Good etags are ones we already know about by finding them in the tag list of an existing diary or by appearing in one of the election roundup diaries. The etag is used to build a more complicated query that looks for the tag in tags, title, or body text (stories/diaries) or in subject and comment text (comments). Certain meta dairies are excluded, including open threads, jotter's daily lists, and sidinny's election roundup, and the top comments diaries.
Software used.The search facility currently installed is based on the swish-e search engine: it is fast (compiled C code), open source, and flexible (you can feed it documents out of a database using your own custom preproccessor). It is intended for document collections numbering a million or less, which will do for Daily Kos with under half a million documents (excluding comments). In addition, swish-e offers many of features most desired, including search of phrases and search within configurable document areas (title, author, tags, URLs). What is indexed. All Daily Kos documents from the beginning to within 5 minutes of the present moment are indexed. Older documents are stored in static yearly indices while more recent documents are indexed and reindexed, frequently at first, then less frequently as they age, until they roll over into a growing, static, permenant yearly index. The right mix of number of index files and frequency of update to obtain the optimal combination of ease of maintenance and speed of search may require further experimentation. At present three different series of indices are maintained, corresponding to (in order of increasing total number of documents) front page stories, user diaries, and comments. For 2005 the number of stories, diaries, and comments was 5097, 88943, and 693224, respectively, containing 56993, 371578, and 2945916 distinct words. Design Considerations. The utility of search to a community of users is delivered not simply by the results of the text search, but most critically by the ordering of the results such that highest "value" hits are displayed first. (Most search researchers know that no one looks beyond about the first handful of results; almost no one looks beyond the first page.) The key to arranging for best to come first is to understand, have access to, and use the meta data associated with the text documents searched. Meta Data. At Daily Kos, we have a number of sorts of meta data that are relevant to documents. First among these is the time of posting. What is the latest? When did they say that? Who was first? That sort of thing. So the one thing search has to get right is time, in the sense of being up to date, that is including the latest diaries, as well as in the sense of permitting searches to be organized by reference to the present time. Making it up to date. Because recommendations are only open for 24 hours after a diary is published, it is important that search indices be refreshed sufficiently often during that interval to provide reasonably up to date information on total recommendations and comments. Currently, diaries up to 6 hours old are refreshed every 5 minutes, and diaries 6 to 24 hours old are refreshed every 15 minutes. More Meta Data. Other important meta data associated with stories and diaries includes comments and recommendations. The number of recommendations offers insight, imperfect, but useful, into the degree of approval a diary has received from the Daily Kos community. The number of comments indicates the size of the discussion engendered. The two numbers together give an interesting insight into a diaries history at Daily Kos. Some user diaries are promoted to the front page, and some diarists ("front pagers") are allowed to post directly to the front page. The latter are not open for recommendation. In order to harmonize listings from searches which include both front page stories and user diaries, a plausible value for number of recommendations (83) is assigned to front page stories, but in lists, this number is indicated by "*" to avoid placing too much emphasis on this made up number. The search query. Search queries can contain individual words, all of which must be found in a text document to constitute a hit. Queries can also contain phrases, indicated by double quotes around the phrase. All words are converted to lower case while searching, so letter case is of no consequence. In addition, queries can include boolean terms, such as "and", "or", and "not" which can be used to construct more complex and perhaps more informative results. By default "and" is assumed between individual words in a query. Finally, queries or parts of queries can be restricted to only search within parts of a document or document metadata. The parts available are shown above, along with examples. Certain characters have special meaning to the search engine (open and close paren, double quote, equals sign). Only a restricted list of characters are expected to occur in a word. For the indices here at daily kos, the word characters are as follows.
All other characters (non word, non special) are silently replaced by spaces while searching. For instance searching with "Jerome a Paris", "Jerome.a.Paris", or "Jerome,a,Paris" all give the same result. This is important in certain cases, for example searching for the url http://riverbendblogspot.com is accomplished with the query site=("riverbend.blogspot.com"). The reasonable looking but incorrect query site=riverbendblog.blogspot.com doesn't work as expected because in the absence of the double quotes and parentheses, replacement of "." with " " and the default boolean AND understood between search words makes it equivalent to site=riverbendblog AND blogspot AND com, which returns as hits only documents with riverbendblog in a url, and blogspot and com in the body text. More on Search
If you are intriqued by search and would
like to know more about the basics of how it is done, this is a
good introduction.
On Search. Basic basics. by Tim Bray. Since I've emphasized the importance of meta data, I'll also point to another article from the same series, On Search. SearchMeta, with some further thoughts on things meta. While I'm on the subject of underlying technologies, let me end this section by pointing to an article by Tim Bray on Ajax, the other new addition here at Daily Kos. In it, he also gives credit to Jesse James Garrett for the Ajax acronym in the process of thoroughly describing it.
Daily Kos is an amazing place, community, phenomenon. It has been my
privilege to have the opportunity to contribute.
-- jotter |
Recommended
- Say whaa? Indeed. Sure, what I’m about to say is anecdotal, but I’m willing to bet my experience in the hours and days since the Republic was outed as an Imperium, has been similar to yours. People who don’t usually pay attention to politics in the...206 comments 266 Recs
- Below is an unrolled string of tweets from Democratic Sen. Sheldon Whitehouse about our political predicament. More of our Senators should be talking like this and joining Whitehouse’s relentless attacks on the corruption and authoritarianism eating...312 comments 699 Recs
- That strike yesterday in Crimea netted big gains — a warehouse full of drones. Bavovna The second house had the ammunition. Ammo Dump Jesus approves of this message. Russia continues to relentlessly shell Kherson. This is a long thread that talks about...59 comments 320 Recs
- I read a story, not here, today saying that the majority decision written by Chief Justice John Roberts which gave immunity to US Presidents for actions which they take while undertaking Official Acts; was based on NOTHING in the Constitution. They...308 comments 636 Recs
- Since the Supreme Court has now ruled that the President of the United States is a king, I’ve figured out what President Biden’s first decree should be. He should declare the electoral college null and void and state that from here on out the President...304 comments 644 Recs
- From the AP: The Biden administration told emergency room doctors they must perform emergency abortions when necessary to save a pregnant woman’s health, following last week’s Supreme Court ruling that failed to settle a legal dispute over whether...31 comments 152 Recs
- with Jill, Hunter, and the Obamas www.borowitzreport.com/… I am posting The Borowitz Report with permission from Andy Borowitz. If you like his work you can see it on SubStack. If you like it a lot, like I do you can subscribe & receive it by email....91 comments 229 Recs
- You can read a blog post about this cartoon, and a transcript, here. I’ll also post the transcript in comments. We can make these cartoons because of hundreds of supporters pledging low amounts - typically $1-$3 - and that's just how I like it! Pretty...30 comments 254 Recs
- Long overdue after they lied regarding Roe and overturning it after saying under oath, Roe would remain in place. These justices are a threat to not only national security regarding bump stocks but this immunity case not even counting Roe. There is...406 comments 801 Recs
- Fascism, Christianity, patriarchy, and white supremacism are on the march in Western Democracies. This time the enemy has been allowed to fester from within, using softer rhetoric than before yet still engendering the same victimhood and hatred. RW...447 comments 354 Recs
- As well as those of the Supreme Court Justices. Just sayin’...167 comments 333 Recs
- Two months ago, Hunter Biden sent a "cease and desist" letter to Fox News demanding that they retract false stories that they have been disseminating for more than a year about President Biden's son. Never mind that the network never had a shred of...42 comments 185 Recs
Hot Tags
- JoeBiden (200)
- DonaldTrump (180)
- SCOTUS (89)
- SupremeCourt (89)
- Recommended (81)
- Election2024 (80)
- Immunity (44)
- Community (43)
- PresidentialDebate (41)
- PresidentialImmunity (30)
- Debate (28)
- Elections (25)
- Election (23)
- Media (23)
- Democrats (23)
- Democracy (20)
- 2024election (15)
- Constitution (14)
- Cartoon (13)
- OpenThread (13)
Recent Diaries
- Do you realize what we’re facing in this country because of Trump and his cult of MAGA ASSHOLES? DO YOU? Unless we get our shit together NOW, and come 100,000,000+ STRONG in November, EXPECT THE WORST. • Concentration Camps that can hold millions of...1 comments 0 Recs
- We started out reading Sherlock Holmes, taking turns reading the stories to each other. Usually at night, before we turned out the light. Then it was Charles Dickens. And finally, Harry Potter — which was a stretch, but it worked out. Chris landed in...2 comments 1 Recs
- Welcome to White House Picture of the Day Showcasing a photo from the White House Photography staff, led by Adam Schultz. Feel free to join us anytime and share with us what is happening in ...1 comments 8 Recs
- This is an “official act”: "Just say the election was corrupt and leave the rest to me and the Republican congressmen." WTF? The right wing SCOTUS majority held that Trump is immune re: claims that he directed the DOJ to conduct knowingly sham...5 comments 8 Recs
- …and this is your brain after processing the events of the last week in American politics: ...or maybe it’s just mine.1 comments 0 Recs
- That old saw about a picture is worth a thousand words? Take a look at this caption map. It clearly shows the pickle the Russians have gotten themselves into in the city of Vovchansk, Kharkiv Oblast, Ukraine. This geolocated map clearly shows that the...8 comments 7 Recs
- For two reasons, One the events occurred BEFORE Trump was President, Two it was in a State Court, which has some protection from Federal interference.6 comments 3 Recs
- Title says it all. I keep seeing people here who’ve fallen for the media myth that Biden can’t step down because that creates a chaotic scenario where non-Harris candidates have a knife-fight for the presidency. The reality is that once Biden endorses...49 comments 2 Recs
- Hello everyone! I have visited Daily Kos several times a day for many years; I finally signed up about a year ago. I've never been a big social media user; totally ignored Facebook for example. I DID have a profile page on Care2 until the pages were...2 comments 1 Recs
- As you can see by Itzl's concerned look, this group gives Kossacks a safe place to check in, a daily diary where we can let people know we are alive, doing OK, ...8 comments 8 Recs
- It’s time to March again — on the Supreme Court — but this time, we won’t be bringing flowers. We’ll be bringing anger! And we have a shitload to be angry about: stripping away the rights of women, the homeless and LGBTQ+ folks, prioritizing guns and...6 comments 6 Recs
- Episode 206 has been uploaded. iHeartRadio this podcast all podcasts Omny.FM this podcast all podcasts ivoox this podcast all podcasts Apple this podcast all podcasts Preview: YouTube, later. Only one block again. SPECIAL COMMENT: “I wasn’t very smart....4 comments 10 Recs
- The Russian Ministry of Defense (MoD) claimed on June 2 that Russian forces conducted an Iskander-M ballistic missile strike against Myrhorod Air Base in Poltava Oblast on June 1 and damaged and destroyed seven Ukrainian aircraft.[78] Former Ukrainian...9 comments 46 Recs
- Please read and follow instructions before opening. Or consult your physician. This is a no flame zone . This vent-hole (hereinafter known as IVH), if lost or stolen will not be replaced or ...19 comments 4 Recs
- Welcome to the Overnight News Digest with a crew consisting of founder Magnifico, regular editors side pocket, maggiejean, Chitown Kev, eeff, Magnifico, annetteboardman, Besame, jck, and �...14 comments 22 Recs
- By now we seemed to have sorted ourselves into camps based on whether or not we think Joe should step aside. Looks like about 10% bed-wetters (here!) and 90% whatever-adjective-I-use-will-get-me-flagged. But at the risk of more ad hominem attacks, let...13 comments 0 Recs
- Reminder: Don’t shoot the messenger. According to reporting bouncing around the net, New Hampshire and New Mexico (!) are in play for Trump, and Biden is losing Georgia and Arizona by double digits. This link is to the Washington Examiner but it’s been...19 comments 3 Recs
- I saw that earlier today Judge Merchan delayed by a couple of months the sentencing of convicted felon, adjudicated rapist, traitor, fraud, and pathological liar Donald J. Trump in the Hush Money case due to the Supreme Court’s historically abusive and...2 comments 10 Recs
- — — MSNBC’s Chris Hayes showed snippets of the Confirmation Hearings of Justices Alito and Roberts, tonight on his show All In. Those clips documented both of the prospective Justices confirming this foundational principle, with regards to the...7 comments 19 Recs
- [after reading, please share your thoughts as comments and participate in a poll that offers as choices a full range of outcomes that these ideas could yield] Last week the Supreme Court overturned the #Chevron decision which was the basis for most...2 comments 2 Recs