Announcing: Slashdot Deals - Explore geek apps, games, gadgets and more. (what is this?)

Thank you!

We are sorry to see you leave - Beta is different and we value the time you took to try it out. Before you decide to go, please take a look at some value-adds for Beta and learn more about it. Thank you for reading Slashdot, and for making the site better!

DARPA Commits To Funding Useful Hacking Projects

timothy posted more than 3 years ago | from the good-work-if-you-can-hack-it dept.

Programming 44

Hugh Pickens writes "Fahmida Y. Rashid reports that the Defense Advanced Research Projects Agency will fund new cyber-security proposals under the new Cyber-Fast Track project intended to cut red tape for hackers to apply for funding for projects that would help the Defense Department secure computer networks, says Peiter Zatko, a hacker known as Mudge who was one of the seven L0pht members who testified before a Senate committee in 1998 that they could bring down the Internet in 30 minutes and is now a program manager for the agency's information innovation office. Anything that could help the military will be considered, including bug-hunting exercises, commodity high-end computing and open software tools and projects with the potential to 'reduce attack surface areas, reverse current asymmetries' are of particular interest. Under the Cyber-Fast Track initiative, DARPA will fund between 20 to 100 projects annually. Open to anybody, researchers can pitch DARPA with ideas and have a project approved and funded within 14 days of the application."

Sorry! There are no comments related to the filter you selected.

Honey Pot? (1, Offtopic)

WrongSizeGlass (838941) | more than 3 years ago | (#37013484)

Could this be a giant honey pot?

49 Year Old Militant Feminist Grandmother Here (0)

Anonymous Coward | more than 3 years ago | (#37013546)

You're an idiot. DARPA has a history of funding technology and has better things to do than make a gigantic trap to destroy their reputation for the purpose of catching a few historically insignificant basement losers.

Re:49 Year Old Militant Feminist Grandmother Here (0)

Anonymous Coward | more than 3 years ago | (#37013600)

.... not sure if just trolled or really stupid ...

Re:49 Year Old Militant Feminist Grandmother Here (0)

Anonymous Coward | more than 3 years ago | (#37013654)

DARPA is a US government agency. The US government has proven to be untrustworthy. I would not put it past the US to misuse DARPA "for the good of the country".

Re:49 Year Old Militant Feminist Grandmother Here (2)

somersault (912633) | more than 3 years ago | (#37013692)

The US government is comprised of humans. Humans have proven to be untrustworthy.

On the other hand, some humans are also trustworthy. Shit. How can I apply both of these into one absurd gross generalisation?

Re:49 Year Old Militant Feminist Grandmother Here (1)

AHuxley (892839) | more than 3 years ago | (#37013750)

Re a few absurd gross historical generalisation?
A 56k using UFO hunter used a perl script to glide around a set of wide open MS "mil" US networks.
The CIA has In-Q-Tel like fronts to seek any useful project at any price and nobody will know.
DARPA has.... like fronts and nobody will know.
The US has usually found solutions to its language, math, computer, crypto, science ect. issues very quickly, with less press and with lots of cash.
Yet now we are to believe the US suffers from unique bug related, surface area and very real "high-end" computing issues.... all very public and only "You" can help..

Re:49 Year Old Militant Feminist Grandmother Here (1)

somersault (912633) | more than 3 years ago | (#37014072)

Well, they did just lose their "AAA" credit rating..

Re:49 Year Old Militant Feminist Grandmother Here (1)

Doc Ruby (173196) | more than 3 years ago | (#37013684)

You mean like the Pentagon did in Iraq?

Re:49 Year Old Militant Feminist Grandmother Here (0)

Anonymous Coward | more than 3 years ago | (#37016126)

This is slashdot. EVERYTHING is a fucking government conspiracy. The government is both completely incompetent and at the same time evil geniuses bent on taking away your open source software.

Re:49 Year Old Militant Feminist Grandmother Here (0)

Anonymous Coward | more than 3 years ago | (#37017554)

shhh, the slashdot bent towards government conspiracies is actually a plot by the government to discredit the users of slashdot!!

Re:49 Year Old Militant Feminist Grandmother Here (0)

Anonymous Coward | more than 3 years ago | (#37020770)

Or maybe that is what they want you to think.

YES (0)

Anonymous Coward | more than 3 years ago | (#37013572)

dont trust

Re:Honey Pot? (0)

Anonymous Coward | more than 3 years ago | (#37013662)

With public disclosure it could never become such.

If DARPA funds hacker x who posts to a list about a vuln in product y at line z, then everybody can see the problem and look towards solutions.

Finally (1)

Anonymous Coward | more than 3 years ago | (#37013588)

For the cost of a few cruise missiles humanity will be left with something of value from the defense budget.

Hopefully this becomes a superfund for cleaning up vulnerabilities by the best and the brightest. With all the money wasted every year we should not agonize over tiny sums being expedited to people who will catch the bugs and disclose to the public.

The cybercrime gangs are well funded. The bughunters are not.

Prevention is much cheaper and much more friendly towards civil liberties than is having a cyberwar bureaucracy staffed by the sort of reactive code grinders who couldn't make it in cutting edge startups.

Mudge? This might actaully work then. (4, Interesting)

sp332 (781207) | more than 3 years ago | (#37013590)

If you recall,there was a campaign to make Mudge the USA cyber-czar back when Obama created that post. The guy knows what he's doing, and even now that he's in big-government stuff, the community still trusts him.

Re:Mudge? This might actaully work then. (0)

Anonymous Coward | more than 3 years ago | (#37014154)

Bating people with skills to make it easier for the government to enforce copy protection laws doesn't endear him. Protecting .mil from attacks is cool and all but that isn't what they are wanting. If it was, it wouldn't be a call to all projects. I find it hard to believe he doesn't realize this. If he doesn't, I hope the clue bus hits him. Soon.

Infinite Military Money (2)

Doc Ruby (173196) | more than 3 years ago | (#37013682)

The military/intel is totally protected from our debt crisis, no matter how distantly related to protecting us any of its expenses might be. That's why the majority of our debt is owed for past military/intel budgets - so it costs 50% more in interest than what was appropriated on paper. And now that the debt has gotten our credit rating downgraded, it will cost us even more in interest - along with all our borrowing that it's dragged along with it.

So the smart people will turn all their projects into military/intel projects. Which will gradually turn the US into not just a hopeless debtor, but an exclusively warmongering hopeless debtor.

Re:Infinite Military Money (1)

NotAGoodNickname (1925512) | more than 3 years ago | (#37013790)

Very true. I know the company I work for loves these programs, there is no risk since it is guaranteed money if you win the contracts. It also diverts engineering resources into supporting these programs.

Re:Infinite Military Money (1)

tryptogryphic (1985608) | more than 3 years ago | (#37013816)

This is why the citizens of any democracy should be on guard, demanding answers from their representatives about spending etc. to ensure that such things do not happen. This is indirect war profiteering in it's finest form.

Re:Infinite Military Money (2)

Doc Ruby (173196) | more than 3 years ago | (#37013838)

Like "why are we invading Iraq when it had nothing to do with the 9/11/2001 attacks"?

Those of us who did ask that question were drowned out by the majority of voters who insisted on re-electing Bush/Cheney instead of impeaching and imprisoning them.

Re:Infinite Military Money (0)

Anonymous Coward | more than 3 years ago | (#37017022)

people deserve the government they get. and people are stupid. the only way to change this is for you, joe citizen, to hold public office and work your way to the top. by the time you get there the system will have molded you into someone no better than the sharks running good ol' USA currently. thems the breaks.

Re:Infinite Military Money (1)

Doc Ruby (173196) | more than 3 years ago | (#37017744)

Well, what's actually more true is that "in a democracy, the people get the government they deserve". Maybe that means we deserve the government we get.

As far as holding public office being the only way to change it, that's clearly not true. Indeed the biggest problem in America's democracy is that our republic, the elected people, are not the ones who make change. They're lackeys to the people who do make the change. And those people are not only the rich. Plenty of not rich (outside their expenses-paid political racket) people fill the ranks of decision makers in the "Social Conservatives" groups that exert such power.

I've actually worked in government, in the NYC City Council (legislature). Change is made by staking out clear and useful positions ahead of the immediate term where the sharks are busy grabbing whatever bleeds. By being persistent, over many years, and playing the social groups to get the access that defines power in politics. And I've also seen some, not many, who get and keep power without being corrupt.

It's pretty broken. But if we just give up and accept the corruption, there's no way out. And we can be much, much worse - look at Argentina, and any of the banana republics we've created in our backyard. Maybe the majority of Americans deserve it - the people who don't even vote, while they see those who do driving us into ditch after ditch. But I deserve better. And I'll do what I can to get it.

Re:Infinite Military Money (0)

Anonymous Coward | more than 3 years ago | (#37017908)

They should have attacked every damn country in the area using targeted air strikes to inflict maximum damage as a clear warning of what would happen the next time some jumped up emir, mullah, or mujahideen done attempts to attack US interests. And also refuse to provide any money after the fact to repair any inflicted damage. There isn't a single country in the world that would interfere. Non-state actors operate under the belief that no matter what atrocity they commit the "international" community will tie themselves in knots politically and won't respond to any provocation. It was assumed that the US could not respond militarily to these types of attacks because the UN would never approve. GWB showed the UN and everyone else that when necessary the US does not really give a shit what other countries think when it comes to US military actions. One very good example of this approach to preventing large attacks is Israels military. The Arab countries have not openly tried to attack the state of Israel in force since their 1973 ass kicking because there is no question in thier minds about what Israel would do for retribution. Of course the Arabs changed tactics and try to use the "palestinians" as cannon fodder and human mortars to avoid direct state attacks and avoid being humiliated once more because of their military ineffectiveness.

Re:Infinite Military Money (1)

Doc Ruby (173196) | more than 3 years ago | (#37017928)

In what area? Iraq? No jumped-up mullah in Iraq or anywhere else except Afghanistan did anything to us on 9/11/2001. GWB showed the UN and the world that even when the US was hideously attacked, all he cared about was invading a country that had nothing to do with it. So his cronies could make $TRILLIONS and grab as much power for as long as they could, while smashing our obligations to protect us. All of which is precisely Binladen and his fellow assholes wanted.

And so you voted for Bush twice, giving us the endless wars, bottomless debts and worthless governments we suffer with now. You Republicans are incapable of learning even the most obvious lessons. Binladen's jihad couldn't have prayed for better partners in the Terror War than you people.

Re:Infinite Military Money (1)

cavreader (1903280) | more than 3 years ago | (#37018596)

Damn near every middle-eastern and N Africa country funds and provides political protection for groups committing terrorists acts all over the world. In a comment above someone said that the people in the US are directly responsible for the consequences of thier government policies so why shouldn't that same principle be true for all countries? This isn't just about 9/11. I am against the wars in Iraq, Libya,and Afghanistan and would have preferred to leave Iraq alone and allow them to continue their own self destruction in peace. And using air based attacks in Afghanistan combined with spec ops Taliban hunters would have been more effective and certainly cheaper. Specifically targeting the vast poppy fields would also hit the Afghan leaders where it hurts and encourage them to actually go after those providing the US with resons to attack. The majority of the US oil imports are from Canada, Mexico, Venezuela, and domestic sources. Let the European countries deal wit the middle east since they rely heavily on Arab oil and they are also the ones responsible for creating this mess in the first place with their arbitrary drawing of country borders in the middle-east. And I didn't vote for Bush. I consider both US political entities ignorant and incompetent of doing anything else besides campaigning. Bin Laden and all of his disciples in jihad have only succeeded in creating a growing hatred of Muslims across the world.

Re:Infinite Military Money (0)

Anonymous Coward | more than 3 years ago | (#37060582)

Nah. After having paid all those people, they will declare the Dollar worthless by printing a quintillion Dollars, and you can wipe your ass with them. Just like China. ^^

Just make sure (0)

Fnord666 (889225) | more than 3 years ago | (#37014310)

Just make sure the funding check clears. It is issued by the US government after all and their credit isn't as good as it once was.

That is all well and good. (1)

das3cr (780388) | more than 3 years ago | (#37014312)

But what is DARPA, or anyone else for that matter, about making sure chips made in china don't have bugs built in?

I /refuse/ to purchase an item that is known to me to have chips made in china because I believe it to be compromised.

How can one be sure that the hardware in the devices made there are not bugged?

Re:That is all well and good. (0)

Anonymous Coward | more than 3 years ago | (#37015022)

"I /refuse/ to purchase an item that is known to me to have chips made in china because I believe it to be compromised." ......so how are you even posting this then?
or are you just very willfully ignorant about where electronics come from?

Non Americans (1)

Yvanhoe (564877) | more than 3 years ago | (#37014666)

But can non-Americans apply ? You know, this category of persons that form 85% of Internet.

Who is scummy enough to work there (-1)

Anonymous Coward | more than 3 years ago | (#37015094)

You would have to be without empathy, and a very much a scum bag to work there, raising your hand to protect the constitution and then turning around and shitting all over it. Until the whole of the corruption in the USA is ejected, and the liars and treasonous fucks are in Ft. Leavenworth, fuck government.

Re:Who is scummy enough to work there (0)

Anonymous Coward | more than 3 years ago | (#37020538)

You can mark my fucking score down so it's un-readable, but you can't change facts.

Social Semantic Desktop for Sensemaking on Threats (1)

Paul Fernhout (109597) | more than 3 years ago | (#37015766)

http://sourceforge.net/projects/pointrel/ [sourceforge.net]

At least I could spin it that way... :-)

And have:
"The need for FOSS intelligence tools for sensemaking etc."
http://groups.google.com/group/openmanufacturing/msg/2846ca1b6bee64e1 [google.com]

Where do I apply? :-)

Re:Social Semantic Desktop for Sensemaking on Thre (1)

Paul Fernhout (109597) | more than 3 years ago | (#37015874)

I see where to apply, a link in one of the articles:
    https://www.fbo.gov/?s=opportunity&mode=form&id=406db188e0e1935a806c143a5603eb48&tab=core&_cview=0 [fbo.gov]

If slashdot allowed longer tittle I woudl have called it: "Social Semantic Desktop for Sensemaking on Threats AND OPPORTUNITIES"

We'll see if they like some variation on:
    http://groups.google.com/group/openmanufacturing/msg/2846ca1b6bee64e1 [google.com]
"Summary: This note is essentially about how civilians could benefit by have access to the sorts of "sensemaking" tools the intelligence community (as well as corporations) aspire to have, in order to design more joyful, secure, and healthy civilian communities (including through creating a more sustainable and resilient open manufacturing infrastructure for such communities). It outlines why the intelligence community should consider funding the creation of such FOSS "dual use" intelligence applications as a way to reduce global tensions through increased local prosperity, health, and with intrinsic mutual security."

Re:Social Semantic Desktop for Sensemaking on Thre (1)

Paul Fernhout (109597) | more than 3 years ago | (#37016386)

I wrote this up last month as a proposal abstract for an IARPA soliciation, but I have not sent it (someone who had been with the CIA and does public intelligence said it would be pointless essentially as the US intelligence community is so broken). Anyway, I though I'd post it here, as I've written it already, and it seems a shame to waste it, and because it is what I'd like to do maybe for this solicitation. Any constructive feedback would be appreciated. Maybe DARPA might be interested in it if not IARPA, given the structural problems in the US intelligence community it seeks to address and which are part of why the US cyber infrastructure is so at risk? Imagine global security researchers having a tool like this to work collectively for mutual benefit to maximize the intrinsic security of our cyber infrastructure. I know some people may say terrible things about any attempt to engage with the US security apparatus (not without some justification), but, beyond being motivated by running out of cash (in part by doing so much free stuff), I do think the issue is that we all need security -- the issue is how we go about getting it. This proposal attempts to shift the US security paradigm in a more intrinsic and mutual direction, which is more sustainable over the long term than a focus on extrinsic (guarded) or unilateral (dominance) security. Maybe others might find the general concept of shifting the security paradigm useful in their own proposals.


Title: "Twirlip: Towards a 21st Century Worldwide Public Intelligence Desktop Platform for Collaborative Sensemaking, Analysis, Risk Assessment, and Horizon Scanning"

Company: Kurtz-Fernhout Software
Organizational form: Woman-owned small business (Cynthia F. Kurtz, CEO)

Prepared: July 12, 2011

Amount requested: US$297,000

Responding to: IARPA Incisive Analysis Office Wide Broad Agency Announcement (BAA) Solicitation Number: IARPA-BAA-10-08, especially these aspects:
* Methods for measuring and improving human judgment and human reasoning
* Understanding and managing massive, dynamic data
* Effective analysis of massive, unreliable, and diverse data
* Assessing relevancy of new data
* Analysis of significant societal events
* Estimation and communication of uncertainty and risk

Summary: As a legacy from the 20th century, there are currently broad institutional barriers in the US intelligence community that make it difficult for intelligence analysts to gain 21st century insights into 21st century issues using 21st century technology and 21st century public data sources. To address the need to move beyond those institutional barriers, we propose a proof-of-concept project called "Twirlip" as a free and open source software (GPL) Public Intelligence desktop platform for the general public. It would use Java/JVM desktop technologies and CouchDB as a backend relay server and indexed archive. It would be built around the idea of a social semantic desktop. The public can then use this system to process open source data to crowdsource sensemaking and analysis about global socioeconomic, technical, and geopolitical trends, with a special emphasis on understanding the likely global consequences of Moore's law. The global community can also expand this platform in various ways by adding new freely licensed modules. The US intelligence community can then build on this public software and public content in its own internal sensemaking and analysis. Supporting this system by IARPA may create both a strategic first mover advantage and a public relations advantage for the US intelligence community. Whether the software is of any use to the US intelligence community directly is not as important as whether the community gets new ideas from seeing what the public does with such tools or seeing how such tools are expanded.

Technical/Administrative contact:
Paul D. Fernhout, CTO
Kurtz-Fernhout Software ...
Website: http://www.kurtz-fernhout.com/ [kurtz-fernhout.com]
Email: pdfernhout@kurtz-fernhout.com

"All of this has the effect of making it hard for DI analysts to interact even with the classified outside world. The CIA view is that there are risks to connecting CIA systems even to classified systems elsewhere. Mitigating those risks sends implicit messages to analysts: that technology is a threat, not a benefit; that the CIA does not put a high priority on analysts using IT easily or creatively; and, worst of all, that data outside the CIA's own network are secondary to the intelligence mission. (http://en.wikipedia.org/wiki/Intelligence_analysis_management#The_work_area )"

"$75 billion a year for secret intelligence, and we still do not have an analytic desktop toolkit, all-source geospatially and historically and cultural astute back office processing, or global reach to all humans, all minds, all the time. Sucks for us. Letâ(TM)s see what the Smart Mob can do⦠(Robert Steele, ex-CIA Officer http://www.phibetaiota.net/2009/10/1988-2009-osint-m4is2-techint-chronology [phibetaiota.net] )"

"The Internet, electronic mail, and the Web have revolutionized the way we communicate and collaborate - their mass adoption is one of the major technological success stories of the 20th century. We all are now much more connected, and in turn face new resulthing problems: information overload caused by insufficient support for information organization and collaboration. For example, sending a single file to a mailing list multiplies the cognitive processing effort of filtering and organizing this file times the number of recipients -- leading to more and more of peoples' time going into information filtering and information management activities. There is a need for smarter and more fine-grained computer support for personal and networked information that has to blend the boundaries between personal and group data, while simultaneously safeguarding privacy and establishing and deploying trust among collaborators. (http://semanticweb.org/wiki/Semantic_Desktop )"

Roy Amara and Ray Kurzweil have both suggested we tend to have an intuitive linear view of change over time but in reality they say technological change tends to be exponential, starting off slow and then cumulatively accelerating. Everything from 3D printers, to robotics and AI, to PV solar energy, to WMD availability, to health care breakthroughs, to the obsolescence of fiat dollars as a means for organizing society may follow this sort of curve. Current US policy-making processes frequently ignore exponential change such as via Moore's Law, whether in:
* thinking the chance US manufacturing jobs might ever significantly return is any more likely than most farming jobs from a century ago will come back;
* assuming that most paid service workers won't be replaced eventually by technological improvements (leading to massive unemployment and a break in the income-through-jobs link underpinning much current US socioeconomics);
* planning for oil drilling, natural gas fracking, and nuclear plants as PV progresses;
* emphasizing unilateral extrinsic security while ignoring the need for mutual intrinsic security as key aspects of the US's defense posture to minimize WMD use risks;
* ignoring the potential for rapidly falling health care costs in the USA from things like people eating more vegetables (Dr. Joel Fuhrman) and getting adequate vitamin D (Dr. John Cannell) that may greatly reduce the incidence of health issues like heart disease, stroke, diabetes, cancer, dementia, allergies, and autism; or
* ignoring that long-term financial plans like a Social Security Trust Fund may be meaningless in an age where everyone could print their own medicine, solar panels, agricultural robots, and elder-care robots, where more and more resource allocations are organized through email, twitters, social media, and robot operating systems.
Many current US policies and related politics shaped in the 20th century are obsolete and irrelevant to 21st century national needs, since the threats and opportunities the US faces now have changed with changes in culture and technology. The US intelligence community has (presumably) not yet supplied the President or Congress with this big picture context or tailored specific advice related to specific global situations from this exponential perspective.

Here is a presentation by our CTO on supporting the initial points on likely US socioeconomic changes with a shifting balance of five types of economic transactions due to cultural change and technological change:
"Five Interwoven Economies: Subsistence, Gift, Exchange, Planned, and Theft"
http://www.youtube.com/watch?v=4vK-M_e0JoY [youtube.com]

A focus by the US intelligence community on stealing secrets instead of making sense of obvious public "open source" (in the intelligence sense) data has led to spending about US$75 billion dollars a year with very little to show for it as far as understanding the big picture of threats and opportunities (according to Robert Steele). As has been suggested (see Wikipedia), because of the worries over secrecy in the CIA and leaks of secret information, technology for processing "open source" (in the intelligence sense) information has become seen by the intelligence community to be more of a threat than an opportunity. Thus one can question if any conventional approach creating yet more secret or proprietary intelligence technology will accomplish much given that culture until organizational barriers to 21st century insights are removed (even as many individual analysts are no doubt brilliant dedicated workers who develop good insights). Individuals can be smart even as organizations can be dumb. How can the US intelligence community as a whole move out of a collective rut before the next huge avoidable disaster befalls the USA? Could the creation of crowdsourced Public Intelligence tools built on a social semantic desktop be part of the answer?

In the sci-fi novel "A Fire Upon the Deep" Vernor Vinge (who coined the term "The Singularity") has a minor character called "Twirlip of the Mists". Twirlip, with a long memory, knows essential information the main characters could have benefited from greatly. Unfortunately the single communication by Twirlip that reached the main characters was ignored due to problems related to weak signal detection including translation issues, communications costs, cultural misunderstandings, and information overload, which together prevent the entirety of Twirlip's communication from helping the main characters' sensemaking and analysis. This project's name is drawn from that example of the problems it seeks to address by supporting improved public and private sensemaking and analysis.

To address these issues, as a proof-of-concept (and a bit more) we will continue the creation of a free and open source (in the software sense) modular social semantic desktop we have been working towards, which will form the seed of a growing Public Intelligence system using open source (in the intelligence sense) data. We are creating a Java/JVM-based desktop frontend with a CouchDB NoSQL backend. Data items are being represented by command messages which define hyperdocuments as a collection of command messages for a specific UUID context that are processed in some order to produce a synthesized document. The software is multi-user and also can operate in a peer-to-peer fashion depending on how CouchDB is configured.

Based on knowledge gained through a decade of research further refined on Genoa II and RAHS, our CEO developed free and open source software called Rakontu that small groups of people can use together to share and work with their stories. It's for people in neighborhoods, families, interest groups, support groups, work groups: any group of people with stories to share. Rakontu members build shared "story museums" that they can draw upon to achieve common goals. Usually people in a Rakontu community will have something they want to do together, and they will be interested in collecting and working with their stories to that end.

Using Rakontu as a core, we feel we can expand that tool to be a more general platform for sensemaking, analysis, risk assessment, and horizon scanning. Milestones along the path include finishing porting our Rakontu software (http://www.rakontu.org/ ) for small group sensemaking and story exchange from Google App Engine to this newer decentralized Java/JVM-based desktop platform, including creating some basic infrastructure for modularity (3 months). Rakontu will provide Twirlip with the core communications tools as documented on that site. The next steps entail building intelligence-related modules for sensemaking and analysis including tools similar in intent to SRI's SEAS and Angler projects for structured arguments and multi-perspective analysis (6 months). In addition, several other social media related applications like wikis, file sharing, outlining, and perhaps rudimentary simulation tools will be integrated in a fine-grained way with this collaborative peer-to-peer system (another 3 months). Throughout that time, sample content will be created for the systems related to global trends including the ones mentioned at the beginning of this abstract and placed under Creative Commons licenses. The goal is a general FLOSS platform the public can use for public intelligence. As it is open, government agencies could add their own modules and data specific to their purposes.

The announcement seeks systems that "provide overwhelming intelligence advantage over its future adversaries". We feel that there is a niche consonant with the national security spirit behind those words by which better sensemaking and analysis, done globally in an open way, can lead to global progress towards intrinsic security and mutual security. As more people adopt an abundance world view instead of a scarcity world view through a more informed understanding of world trends, that may make the USA more secure overall. That global mindshift may reduce the risk that the USA will be involved in conflicts where it would need to rely on extrinsic or unilateral security. This shift is part of a move to a post-scarcity abundance-oriented socioeconomic model we predict will increasingly come to represent a 21st century enlightenment, one that moves beyond the irony of technologies of abundance in the hands of those thinking in terms of scarcity. As with the name of the project, this key message may well be ignored, but we feel we have to try to present it anyway.

Our company is a woman-owned small business run by Cynthia Kurtz, who has previously worked on narrative sensemaking technology for the Genoa II project and Singapore's RAHS project (including contributions to design discussions about Angler). Her husband, Paul Fernhout, is an accomplished programmer who has helped her in the past on a variety of projects including those two, and whose undergraduate work with George A. Miller at Princeton was one of the inspirations for WordNet. Both have been listed in the "Who's Who" of Public Intelligence:
    http://www.phibetaiota.net/2011/06/whos-who-in-public-intelligence-cynthia-kurtz/ [phibetaiota.net]
    http://www.phibetaiota.net/2011/06/whos-who-in-public-intelligence-paul-fernhout/ [phibetaiota.net]
In order to move this freely licensed version forward (under the GPL or a similar free software license) drawing from the published literature on intelligence techniques, we will put in 1.5 FTEs between us (0.5 FTE for Cynthia Kurtz and 1.0 FTE for Paul Fernhout) to develop new code and content, at a cost for the one year project including profit and overhead at US$297,000. Because the project is free and open source (in the software sense) and will be developed completely in the open, we hope other individuals (potentially from any country) will contribute modules, but we can not guarantee that. After the concept is proven, as measured by the number of global users, future follow-up activities could include developing a variety of more complex additional modules such as translation support, geospatial awareness, advanced data importing routines, complex agent-based simulation tools, spreadsheets, and so on. Long term, this project might require tens of millions of dollars a year from a variety of funding sources and thousands of person-years of effort by both paid staff and volunteers to operate on the scale of Wikipedia, to finally realize Doug Engelbart's and other's visions. What we propose here is getting that snowball rolling.

This system will hopefully help the US public as well as the US intelligence community better predict and act on long term trends. Our CTO has been accurate in predicting some aspects of big trends based on Moore's law even without such tools, but better tools might help increase precision and confidence in predictions. For example, here is a copy of an email he sent to DARPA where he sadly predicted aspects of 9/11/2001 in 1999 (predicting a small unstable group figuring out how to use technology as an amplifier to make WMDs, suggesting that issue will only grow over time): http://groups.google.com/group/virgle/msg/64c7c2fb922a4bcf [google.com] Based on Moore's law interpreted broadly, he predicted in 2000 a variety of current ongoing breakthroughs by 2020 from PV becoming cheaper than coal, to dexterous robotics, to the rise of social media: http://dougengelbart.org/colloquium/forum/discussion/0126.html [dougengelbart.org]
Here he essentially predicted the $100 OLPC project and smartphones based on Moore's law: http://dougengelbart.org/colloquium/forum/discussion/0754.html [dougengelbart.org]
However, all predictions are subject to much uncertainty, and no one is right all the time. In these predictions he was right about trends, not about specifics. While being right about specifics in real-time may help put out emerging fires, being right about trends in advance can help in reducing the likelihood fires will start. So, even getting an imprecise sense of trends can help in making better policy decisions. Unfortunately, such predictive power based on taking Moore's law and exponential change seriously has not yet spread widely into public policy decision making. The hope is that better tools used collectively by the public (as well as the intelligence community) will lead to better analysis of 21st century global trends, better predictions, and better proposals for dealing with them in a constructive healthy way by a 21st century USA.

Advantages to intelligence agencies of supporting this dual-use platform include:
* The intelligence community will have a non-secret training tool useable by recruits for skill development before they have security clearances (DI's "America's Army"?).
* The US intelligence community may gain some public relations (PR) advantage through demonstrating value to many educated US Americans in a tangible way by having funded software they use regularly, given many of the community's successes can not be discussed.
* The public will be crowdsourcing analysis of trends and related open data sets that intelligence analysts would be able to use in various ways if they are published.
* To the extent these tools are adopted by other countries (given they are free), the USA will have an understanding of what tools and possibly datasets other countries are using for intelligence with what specific strengths and weaknesses while maintaining relative advantage through its own extra secret/proprietary tools.
* There may eventually be a related reduction in global tensions by better local decision making so intelligence analysts have less overwhelming amounts of data to process and so may be better able to detect weak signals.
* Such tools will be useable for improving public health through better sensemaking and analysis related to health care studies and models, making the USA more secure through a healthier population and may create a more intelligent population better able to cope with emerging threats and opportunities (as health affects intelligence) .
* The intelligence community will be able to watch what novel ideas for sensemaking and analysis are created through through global cooperation, some of which may then be useful for further in-house development along specialized lines.
* These tools may be more easily accessible to people in other US government agencies like the Department of Agriculture, the NIH, the NSF, the State Department, and so on, as well as by Congressional staff, to increase the level of informed productive sensemaking and analysis of trends by US government staff beyond that being done by people with Secret clearance.

Other related documents by our CTO on using post-scarcity-oriented public intelligence to build a world that is safer, happier, and healthier for US citizens include:
http://pcast.ideascale.com/a/dtd/76207-8319 [ideascale.com]
http://groups.google.com/group/openmanufacturing/msg/2846ca1b6bee64e1 [google.com]
http://slashdot.org/comments.pl?sid=1897006&cid=34459370 [slashdot.org]
http://www.pdfernhout.net/post-scarcity-princeton.html [pdfernhout.net]
http://www.pdfernhout.net/on-dealing-with-social-hurricanes.html [pdfernhout.net]
http://www.pdfernhout.net/recognizing-irony-is-a-key-to-transcending-militarism.html [pdfernhout.net]

From the last link: "Likewise, even United States three-letter agencies like the NSA and the CIA, as well as their foreign counterparts, are becoming ironic institutions in many ways. Despite probably having more computing power per square foot than any other place in the world, they seem not to have thought much about the implications of all that computer power and organized information to transform the world into a place of abundance for all. Cheap computing makes possible just about cheap everything else, as does the ability to make better designs through shared computing. ... There is a fundamental mismatch between 21st century reality and 20th century security thinking. Those "security" agencies are using those tools of abundance, cooperation, and sharing mainly from a mindset of scarcity, competition, and secrecy. Given the power of 21st century technology as an amplifier (including as weapons of mass destruction), a scarcity-based approach to using such technology ultimately is just making us all insecure. Such powerful technologies of abundance, designed, organized, and used from a mindset of scarcity could well ironically doom us all whether through military robots, nukes, plagues, propaganda, or whatever else... Or alternatively, as Bucky Fuller and others have suggested, we could use such technologies to build a world that is abundant and secure for all."

It is only a matter of time before someone somewhere creates such a system for Public Intelligence, given advancing technology and social networks. The question is, does the US intelligence community want to try to gain a first mover advantage and possibly a public relations advantage by being involved with such a system's creation?


I have a brilliant idea! (1)

SendBot (29932) | more than 3 years ago | (#37015814)

I'm going to seek a $20k grant to advise police agencies against having their website developed by BJM marketing.

In case you are wondering what the hell I'm talking about: http://www.computerworld.com/s/article/9218961/AntiSec_hackers_dump_data_after_hacking_police_websites [computerworld.com]

Why not insist on free and open source? (1)

Paul Fernhout (109597) | more than 3 years ago | (#37015962)

From the reuters article: "Addressing a key issue for hackers doing government projects, they will be allowed to keep the commercial intellectual property rights while giving the Defense Department use of the project."

Major problem with entire solicitation design (1)

Paul Fernhout (109597) | more than 3 years ago | (#37016140)

I skimmed through the solicitation. It has people paid on achieving milestones they set out in advance (and they say ideally for two month or four month working time frames). Essentially, they are insisting on a waterfall development model. That makes difficult any basic research and general creativity in exploring topic areas. I guess someone could get around that a bit by promising a report or something, but that is probably not what they are looking for.

In general it is a rule of thumb in some projects by competent people that those who do not promise delivery dates get done faster. :-)

It's not clear to me how streamlined this is relative to usual government proposals, other than a quicker approval turnaround and shorter project scopes. You still need to do a bunch of paperwork and planning.

For what I want to do, with a social semantic desktop that does some specific things for public sensemaking, where I've worked on related stuff for years, and made some related stuff like that before (for governments), there may be just enough potential for milestone definition for some proposal. I could see some other people might have projects they've long been wanting to do and worked on pieces of that they could try to fit into this too. But for most people, thinking of something new, it would not be easy to plan for those milestones if they were other than work for X hours, and the endeavor could be high risk for the proposer if they don't meet their milestone (they would presumably not get paid?). Anyway, I just skimmed it, so maybe I missed something.

I'd suggest DARPA might have more success if they just asked for resumes from talented people and small groups, said we will fund you to work wherever (home office) for three months on cool free and open source stuff in an area you propose and we find interesting related to security, and if you want more funding after that, we'll decide based on what you deliver in that time period. Call the program "DARPA Cyber-Security Fellows" or something like that.

I'd be curious what other have to say on that.

Re:Major problem with entire solicitation design (0)

Anonymous Coward | more than 3 years ago | (#37019734)

Complete project out of curiosity.
Feed "milestones" while working on next project.

You just need a 1 project buffer!

(just a joke please don't let darpa get me)

Re:Major problem with entire solicitation design (1)

Paul Fernhout (109597) | more than 3 years ago | (#37021738)

It's a joke with lot of truth to it. My undergrad adviser said he used this model sometimes (he's 90 or so now, so probably OK to mention this). He said he would essentially get a grant for work he had already (mostly) done, and then use much of the money to do the next thing. So, you are right, it's an interesting and sometimes successful model.

A much deeper problem is that the people good at looking good may not be the same people good at doing stuff. As someone suggested recently (forget where, maybe on slashdot) that is why so many mediocre films are produced. The best directors and writers may not be the best at convincing others to give them money to make films. This is in part a function of how many lesser skilled wannabees are around and how desirable the area is. The more mature a field is, perhaps the bigger the problem?

I think that was implied in another recent slashdot article that at first glance seemed to be about how the popularity of computer programming was insuring the unemployment of true geeks. Will a true geek, even one with decent social skills, get hired when hiring managers can find a lot of very appealing people who look even more on paper like true geeks than the true geeks, and they can't tell the difference, or at least, can't tell from the information they have to work with? This is also a problem in the "Seven Samurai", how does a farmer know what makes a good Samurai? And there are so many aspects to what makes people effective, even a focus on skills and experiences can be misleading.

A completely different issue is you may be hiring the wrong type of person, or the wrong person may be doing the hiring. For example, this presentation by David Eaves suggests that big open source projects need good facilitators at the core more than they need good coders:
    http://www.slideshare.net/david_a_eaves/community-management-presentation/ [slideshare.net]

Still, coding skills in the case of open source may be important for a certain level of respect by the community. In general, we need better software tools for collaboration, as that presentation talks about (and thus the need for a social semantic desktop and good tools on it, including for stuff like Structured Dialogic Design and a variety of other methods for collective sensemaking and analysis and collaboration).
    http://www.globalagoras.org/ [globalagoras.org]
    http://en.wikipedia.org/wiki/Sensemaking [wikipedia.org]
    http://collaboration.wikia.com/wiki/Stigmergic_collaboration [wikia.com]

The best "manager" I ever had in a commercial setting did not know how to code that well (although he could code enough to understand the problem area and contribute to it), but he was great at managing a team well.

Another option for running a program like this is to not have applications. Just find people doing the work you like and give them money.

Still, ultimately, the best security is going to emerge from a society with things like a "basic income" to live off of so the people who like resolving these issues have the time to do so, without imposing this problematical filtering process on it. That is what is depicted in James P. Hogan's "Voyage From Yesteryear" sci-fi novel. And it is backed up by research, like discussed here:
    "RSA Animate - Drive: The surprising truth about what motivates us "
    http://www.youtube.com/watch?v=u6XAPnuFjJc [youtube.com]

The best motivated work comes from taking money off the table, and people having a sense of purpose, developing a sense of mastery, and having a sense of ownership/influence over what is happening.

This is all why it is so how hard to give money away well, as discussed near the end to the Seven Laws of Money book by Michael Philips. Giving away money well is a tough problem.

With that said, I don't know much specifically about the field of security vulnerabilities. So there may well be projects and people this program has in mind already that are good matches for the money.

Actually, just the other day I was thinking (as I sometimes do, on and off) on how, given Bruce Schneier's points of "Security Through Simplicity" http://www.schneier.com/essay-018.html [schneier.com] that it would be cool to build to build secure handhelds that were based on the simplest Forth-based hardware ICs (maybe as a parallel set on silicon), because Forth hardware and code is easy too make, with a Smalltalk-like message passing system on top of that, using a hardware random number generator to make secure one-time pads which would be exchanged in person. A social semantic desktop could then be put on top of that, based around shared streams of command messages that manipulate data structures (ones often made of triples).

One starting reference:
    http://www.ultratechnology.com/forth.htm [ultratechnology.com]
"In this world there are few people working on making computers simple to understand, simple to build, and simple to program. There are few people making programs that are easy to understand, easy to maintain, efficient and beautiful. One of those people is Charles Moore the inventor of the computer language Forth. Chuck Moore describes himself as a professional who gets personal satisfaction out of seeing a job done well. He enjoys designing computers and writing very efficient software. He has been working for nearly thirty years to come up with better software and nearly twenty years to come up with better computer hardware. His latest work involves unusually small computers both in hardware and software. His ideas are very synergistic as both his hardware and software are as much as 1000 times smaller than conventional hardware and software designs. However many of his ideas about software design and programming style are not limited to his tiny machines. While it is difficult to map bloated software techniques to tiny machines it is easy to map his tight tiny software techniques to huge machines. There will always be problems bigger than our machines and there will always be people who want to get the most out of their hardware, their software, and their own productivity."

We used to have real-time multi-user hardware using Forth with only a few K of memory. What happened? If we want to scale up, why not scale in a parallel direction, not with more complexity in each node?

That is the ultimate in secure computing, because it is all very simple, and the more complex part is inspectable. (I'm not saying there might not be complex emergent effects.)

But I mostly do software because it is a lot cheaper than messing with hardware like this (such a project would cost millions to make for real, including working with fabs, although you could prototype it with FPGAs or even just transistors for one core, it is so simple). And who would fund me to make hardware/software like this without a recent track record in hardware? And in any case, it would be best to make such a thing as a team. So, there is a great project about security, but organizing it is not so easy.

There are aspects of this that tie in with Alan Kay's "Fundamentals of New Computing (FONC) project as well.

Also, I did not refine this sort of idea (hazy as it still is) on a "grant" but on my own time, over many years. Which is another part of how the granting process is broken. Another dimension on this which grants can have trouble cutting through:
    http://disciplinedminds.com/ [disciplinedminds.com]
"In this riveting book about the world of professional work, Jeff Schmidt demonstrates that the workplace is a battleground for the very identity of the individual, as is graduate school, where professionals are trained. He shows that professional work is inherently political, and that professionals are hired to subordinate their own vision and maintain strict "ideological discipline." [And have only "assignable" curiosity.] The hidden root of much career dissatisfaction, argues Schmidt, is the professional's lack of control over the political component of his or her creative work. Many professionals set out to make a contribution to society and add meaning to their lives. Yet our system of professional education and employment abusively inculcates an acceptance of politically subordinate roles in which professionals typically do not make a significant difference, undermining the creative potential of individuals, organizations and even democracy."

And also:
    http://p2pfoundation.net/Towards_a_Free_Matter_Economy [p2pfoundation.net]
""Grants provide money in advance, when it is most needed, both to provide for the material needs of the project and to pay researchers for their time, making them the most obvious way to fund any public good. We've funded university and institutional research this way for ages. Many precursors of free software products got started like this, including the code that became the Unix family of operating systems. Clearly this system can and does work. However, grant money, once given, is extremely hard to get back. So when a person applies for a grant they must endure a gauntlet of tests, intended to prove to the granting agency that that person is willing and able to fulfill the promise they make in their proposal. If a grant is received, will the project be completed? How much money will it cost? Real research is full of unexpected set-backs and cost-overruns. Real researchers are full of optimism and unrealistic deadlines. The skills for research, development, logistics, and management are rarely found all in one person -- good scientists rarely make good accountants, let alone good receptionists. This encourages the granting agency to be very selective and take few risks with whom they fund. Researchers must have a proven professional background, track-record of honesty, and a reputation to protect. This is why funding by grants requires the use of large government, foundation, and university bureaucracies, and only the professionals who have climbed the career ladder to positions in these organizations have a serious chance of benefitting from them. The "solitary inventor" is indeed dealt out of this game, just as Eisenhower predicted." ( http://www.freesoftwaremagazine.com/articles/free_matter_economy?page=0%2C1 [freesoftwaremagazine.com] ) "

This is all why I tend to think places more like Willow Garage are the future of innovation. Long term funding allowing the assembling of teams of good people working in an open way. Still, a basic income for the USA would allow those teams to come together on their own. But until then, a place like Willow Garage is a good compromise. SRI and RAND had aspects of this in the past, but they were never open, did not have patient money, and always chose to "double dip" at the public trough, first to get the grants and then to sell the results.

All of this is reasons the whole US security model (beyond intelligence) is deeply broken. And as we just saw in US politics, the broken socioeconomic model in the USA is starting to have serious implications (like bond downgrades). Well, what is going to be the implication of a broken socioeconomic model (relative to the 21st century needs) on US defense? The country will not be able to marshall its best talents together to create true (intrinsic, mutual, sustainable) security, and the countries defenses will fail relative to other countries that can.

I think there is still hope for the USA, but the fact is, if it had not started out so rich and with so many advantages a century ago, it would have been in much worse crises than Greece and Italy etc. long ago. I'm not saying China is better at innovation than the USA could be, but this comparison shows how messed up the USA has become:
    http://www.nytimes.com/2009/09/09/opinion/09friedman.html [nytimes.com]
"Watching both the health care and climate/energy debates in Congress, it is hard not to draw the following conclusion: There is only one thing worse than one-party autocracy, and that is one-party democracy, which is what we have in America today.
    One-party autocracy certainly has its drawbacks. But when it is led by a reasonably enlightened group of people, as China is today, it can also have great advantages. That one party can just impose the politically difficult but critically important policies needed to move a society forward in the 21st century. It is not an accident that China is committed to overtaking us in electric cars, solar power, energy efficiency, batteries, nuclear power and wind power. China's leaders understand that in a world of exploding populations and rising emerging-market middle classes, demand for clean power and energy efficiency is going to soar. Beijing wants to make sure that it owns that industry and is ordering the policies to do that, including boosting gasoline prices, from the top down.
    Our one-party democracy is worse. The fact is, on both the energy/climate legislation and health care legislation, only the Democrats are really playing. With a few notable exceptions, the Republican Party is standing, arms folded and saying "no." Many of them just want President Obama to fail. Such a waste. Mr. Obama is not a socialist; he's a centrist. But if he's forced to depend entirely on his own party to pass legislation, he will be whipsawed by its different factions."

Anyway, so that is why you and I are "joking" about how to game the system -- because the core socioeconomic model has failed.

Another reason the grant process has failed:
    http://www.its.caltech.edu/~dg/crunch_art.html [caltech.edu]
"Peer review is usually quite a good way to identify valid science. Of course, a referee will occasionally fail to appreciate a truly visionary or revolutionary idea, but by and large, peer review works pretty well so long as scientific validity is the only issue at stake. However, it is not at all suited to arbitrate an intense competition for research funds or for editorial space in prestigious journals. There are many reasons for this, not the least being the fact that the referees have an obvious conflict of interest, since they are themselves competitors for the same resources. This point seems to be another one of those relativistic anomalies, obvious to any outside observer, but invisible to those of us who are falling into the black hole. It would take impossibly high ethical standards for referees to avoid taking advantage of their privileged anonymity to advance their own interests, but as time goes on, more and more referees have their ethical standards eroded as a consequence of having themselves been victimized by unfair reviews when they were authors. Peer review is thus one among many examples of practices that were well suited to the time of exponential expansion, but will become increasingly dysfunctional in the difficult future we face."

So, the choice is, does the US want to take a risk on a 21st century defense apparatus and the socioeconomics that goes with that, or does it want to hope a 19th century defense apparatus and socioeconomics will keep working well enough?

As someone concerned about the nested series of institutions I find myself in from town to state to nation to planet to solar system (one might call that loyalty or patriotism of a certain sort), I tried to upgrade that paradigm over the years:
    http://groups.google.com/group/virgle/msg/64c7c2fb922a4bcf [google.com]
    http://www.pdfernhout.net/recognizing-irony-is-a-key-to-transcending-militarism.html [pdfernhout.net]

But with little apparent success other than I'm gradually appearing less and less weird, even though I have not changed that much. :-)

As I've said before, I see to often in some areas of my interests find myself a generation or two behind the real though leaders (Einstein, Illitch, Lovins, Moore, Taylor, Dyson, O'Neill, Hogan, Sturgeon, Kay, Engelbart, Licklider, etc.) and a generation ahead of the mainstream. I guess that's what comes of reading so much sci-fi as a kid. :-)

But the value perhaps in that spot is being able to put together the ideas of the previous generation in new ways, and hope the next generation gets that (assuming it is still relevant). Pity there is not more funding for that (or a basic income). This particular program is maybe a step towards that, but there just is some much bigger gap in that area. Seriously, it would be *cheap* for the USA to print ten trillion dollars and pour it into exploring sustainable mutual intrinsic security. Dirt cheap, compared to the alternative, the end of the infinite game.
    http://en.wikipedia.org/wiki/Finite_and_Infinite_Games [wikipedia.org]

But somehow I doubt that level of investment will happen, at least not now when it would make a difference (and create a bunch of jobs, too). But whereas in the past social collapse has been a way forward for broadly dysfunctional systems, that is more problematical when the word and the USA especially has so many WMDs ready to go on minutes notice:
    http://en.wikipedia.org/wiki/Societal_collapse [wikipedia.org]

It seems to me that the best hope the USA has for security at this point is that much of the rest of the world is more compassionate to it than it has been to much of the rest of the world over the past few decades (Korea, Vietnam, Chile, Iraq, etc.). Still, there remains some warm feelings from some good things in the past, and I think an argument can be made, same as for the Romans, that a dysfunctional empire overall was better than constant local feuding (especially with modern WMDs). So, if there is a reasons to cut the USA some slack for past misdeeds, that might be it. It's true the world has not yet blown itself up with escalating tensions somewhere. But whatever past "successes" can be claimed, that can't justify not going forward in a better way now, given the internet and automation and better design, because another better joyful world that works for (almost) everybody (and has mutual and intrinsic security for all) is clearly within sight.

And always remember, the physical security of most US citizens depends on the proper functioning of 1970s Soviet-era computer technology (that the US tried to sabotage) controlling ICBMs. That one fact should cause US defense planners to question most of their assumptions about "security", IMHO. Let alone what happens when kids can download plagues from the internet (we think computer viruses are bad...)

Still, this program is hopeful in that it is another step for DARPA towards a 21st century security model, and so it is to be applauded and encouraged in that sense. And it is not easy to make anything happen in a big bureaucracy, so the people who brought things this far are to be congratulated on getting something going. They are trying.

The deep deep problem is that the US DOD has two conflicting missions. One is the stated mission, to defend the country physically (and the constitution, of course). The other is the unstated mission, which is to defend the particular current socioeconomic hierarchy in the country right now. Those two currently conflict in a huge way, and so the DOD itself is a warzone. Related about this conflict, which goes back a long time, by a very decorated Marine Major General:
    http://www.lexrex.com/enlightened/articles/warisaracket.htm [lexrex.com]

That said, we all need security. The issue is how we go about getting it in a healthy and non-ironic way.

Hudson says.... (1)

meglon (1001833) | more than 3 years ago | (#37016210)

Sarge, is this going to be a stand up fight, or just another bug-hunt?

Re:Hudson says.... (1)

gmhowell (26755) | more than 3 years ago | (#37018680)

There may be a xenomorph involved.

Social Media Protocals (1)

hhawk (26580) | more than 3 years ago | (#37018564)

Open Protocols for Social Media would be very helpful..

Think along the lines of Diaspora and Google+ but within a military context, where each command/outfit, etc. needs to own it's own data, various aspects of data needs to shared (nor not shared) based on a firm but flexible set of permissions and you have a fairly ideal way of allowing modern war fighting use social tools; all of those still on secure networks but having a wide range of secure sharing. This could include pushing data out to non secure networks from civilian to governmental (e.g, congress, white house, etc.) and NGOs or pushing data to other secure networks (e.g., CIA, NSA, etc.).


I'm submitting this... apk (0)

Anonymous Coward | more than 3 years ago | (#37022146)

A return to the "old" to combat the problems of "the new" & why, in combination with filtering DNS servers (vs. malware-in-general in most ALL forms) that use DNSBL's vs. them! I have done so for YEARS now (since 2002 in my older Delphi model, which used "brute force" dedup methods which was FINE on HOSTS files in those days that only MAYBE hit 16k lines - lately, they're a LOT larger than that, so I switched to a Python system my nephew & I co-wrote that processes MILLIONS @ a time & faster dedup algorithms in place is why because of Python's built in routines).

It does the following things:


1.) Data gather from reputable sources for HOSTS data (some listed below, not all though), DNSBL's too!

2.) Alphabetize the data

3.) Removes duplicates/normalizes the data

4.) Changes from the larger & slower "loopback adapter address" to the just as compatible & faster "blackhole routing" address instead

5.) Filtering vs. "problematic" sites that MAY 'disturb' some sites IF their adbanner servers are disrupted (YAHOO, AOL, MSN & quite a few others)

6.) Commits back (from a "temp/scratch" file) to the ORIGINAL HOSTS file for use by the system &/or apps (@ RPL 0/Ring 0/kernelmode level, FAR faster & more efficient than Ring 3/RPL 3/Usermode filtering solutions are mind you) by OVERWRITE, assuring CLEAN COPY & a pristine unaltered (by malware) HOSTS file!


As well as a recommendation for this, in combination with it (using the excellent CIS Tool as a guide) -> http://www.bing.com/search?q=%22HOW+TO+SECURE+Windows+2000%2FXP%22&go=&form=QBRE [bing.com]

My custom HOSTS file currently protects me vs. 1,554,666++ (& growing every 15 minutes) KNOWN bad sites/servers/hosts-domains that are KNOWN to be either maliciously scripted, or serving up malware-in-general, plus spamming/phishing sources as well as botnet C&C servers.

How/Why? Ok, read on:


1.) HOSTS files are useable for all these purposes because they are present on all Operating Systems that have a BSD based IP stack (even ANDROID) and do adblocking for ANY webbrowser, email program, etc. (any webbound program).

2.) Adblock blocks ads in only 1-2 browser family, but not all (Disclaimer: Opera now has an AdBlock addon (now that Opera has addons above widgets), but I am not certain the same people make it as they do for FF or Chrome etc.).

3.) Adblock doesn't protect email programs external to FF, Hosts files do. THIS IS GOOD VS. SPAM MAIL or MAILS THAT BEAR MALICIOUS SCRIPT, or, THAT POINT TO MALICIOUS SCRIPT VIA URLS etc.

4.) Adblock won't get you to your favorite sites if a DNS server goes down or is DNS-poisoned, hosts will (this leads to points 5-7 next below).

5.) Adblock doesn't allow you to hardcode in your favorite websites into it so you don't make DNS server calls and so you can avoid tracking by DNS request logs, hosts do (DNS servers are also being abused by the Chinese lately and by the Kaminsky flaw -> http://www.networkworld.com/news/2008/082908-kaminsky-flaw-prompts-dns-server.html [networkworld.com] for years now). Hosts protect against those problems via hardcodes of your fav sites (you should verify against the TLD that does nothing but cache IPAddress-to-domainname/hostname resolutions via NSLOOKUP, PINGS, &/or WHOIS though, regularly, so you have the correct IP & it's current)).

6.) Hosts files don't eat up CPU cycles like AdBlock does while it parses a webpages' content, nor as much as a DNS server does while it runs. HOSTS file are merely a FILTER for the kernel mode/PnP TCP/IP subsystem, which runs FAR FASTER & MORE EFFICIENTLY than any ring 3/rpl3/usermode app can.

7.) HOSTS files will allow you to get to sites you like, via hardcoding your favs into a HOSTS file, FAR faster than DNS servers can by FAR (by saving the roundtrip inquiry time to a DNS server & back to you).

8.) AdBlock doesn't let you block out known bad sites or servers that are known to be maliciously scripted, hosts can and many reputable lists for this exist:


http://www.mvps.org/winhelp2002/hosts.htm [mvps.org]
http://someonewhocares.org/hosts/ [someonewhocares.org]
http://hostsfile.org/hosts.html [hostsfile.org]
http://hostsfile.mine.nu/downloads/ [hostsfile.mine.nu]
http://hosts-file.net/?s=Download [hosts-file.net]
https://zeustracker.abuse.ch/monitor.php?filter=online [abuse.ch]
https://spyeyetracker.abuse.ch/monitor.php [abuse.ch]
http://ddanchev.blogspot.com/ [blogspot.com]
http://www.malware.com.br/lists.shtml [malware.com.br]
http://www.stopbadware.org/ [stopbadware.org]
Spybot "Search & Destroy" IMMUNIZE feature (fortifies HOSTS files with KNOWN bad servers blocked)

And yes: Even SLASHDOT &/or The Register help!

(Via articles on security (when the source articles they use are "detailed" that is, & list the servers/sites involved in attempting to bushwhack others online that is... not ALL do!)).

2 examples thereof in the past I have used, & noted it there, are/were:

http://it.slashdot.org/comments.pl?sid=1898692&cid=34473398 [slashdot.org]
http://it.slashdot.org/comments.pl?sid=1896216&cid=34458500 [slashdot.org]

9.) AdBlock & DNS servers are programs, and subject to bugs programs can get. Hosts files are merely a filter and not a program, thus not subject to bugs of the nature just discussed.

10.) HOSTS files protect you vs. DNS-poisoning &/or the Kaminsky flaw in DNS servers, and allow you to get to sites reliably vs. things like the Chinese are doing to DNS -> http://yro.slashdot.org/story/10/11/29/1755230/Chinese-DNS-Tampering-a-Real-Threat-To-Outsiders [slashdot.org]

11.) HOSTS files are EASILY user controlled, obtained (for reliable ones -> http://www.mvps.org/winhelp2002/hosts.htm [mvps.org] ) & edited too, via texteditors like Windows notepad.exe or Linux nano (etc.)

12.) With Adblock you had better be able to code javascript to play with its code. With hosts you don't even need source to control it (edit, update, delete, insert of new entries via a text editor).

13.) Hosts files are easily secured via using MAC/ACL &/or Read-Only attributes applied.

14.) Custom HOSTS files also speed you up, unlike anonymous proxy servers systems variations (like TOR, or other "highly anonymous" proxy server list servers typically do, in the severe speed hit they often have a cost in) either via "hardcoding" your fav. sites into your hosts file (avoids DNS servers, totally) OR blocking out adbanners - see this below for evidence of that:


US Military Blocks Websites To Free Up Bandwidth:

http://yro.slashdot.org/story/11/03/16/0416238/US-Military-Blocks-Websites-To-Free-Up-Bandwidth [slashdot.org]

(Yes, even the US Military used this type of technique... because IT WORKS! Most of what they blocked? Ad banners ala doubleclick etc.)


Adbanners slow you down & consume your bandwidth YOU pay for:

ADBANNERS SLOW DOWN THE WEB: -> http://tech.slashdot.org/article.pl?sid=09/11/30/166218 [slashdot.org]


And people do NOT LIKE ads on the web:

PEOPLE DISLIKE ADBANNERS: http://yro.slashdot.org/yro/08/04/02/0058247.shtml [slashdot.org]


As well as this:

Users Know Advertisers Watch Them, and Hate It:

http://yro.slashdot.org/yro/08/04/02/0058247.shtml [slashdot.org]


Even WORSE still, is this:

Advertising Network Caught History Stealing:

http://yro.slashdot.org/story/11/07/22/156225/Advertising-Network-Caught-History-Stealing [slashdot.org]


15.) HOSTS files usage lets you avoid being charged on some ISP/BSP's (OR phone providers) "pay as you use" policy http://yro.slashdot.org/story/10/12/08/2012243/FCC-Approving-Pay-As-You-Go-Internet-Plans [slashdot.org] , because you are using less bandwidth (& go faster doing so no less) by NOT hauling in adbanner content and processing it (which can lead to infestation by malware/malicious script, in & of itself -> http://apcmag.com/microsoft_apologises_for_serving_malware.htm [apcmag.com] ).

16.) If/when ISP/BSP's decide to go to -> FCC Approving Pay-As-You-Go Internet Plans: http://yro.slashdot.org/story/10/12/08/2012243/FCC-Approving-Pay-As-You-Go-Internet-Plans [slashdot.org] your internet bill will go DOWN if you use a HOSTS file for blocking adbanners as well as maliciously scripted hacker/cracker malware maker sites too (after all - it's your money & time online downloading adbanner content & processing it)

Plus, your adbanner content? Well, it may also be hijacked with malicious code too mind you:


Ad networks owned by Google, Microsoft serve malware:

http://www.theregister.co.uk/2010/12/13/doubleclick_msn_malware_attacks/ [theregister.co.uk]


Attacks Targeting Classified Ad Sites Surge:

http://it.slashdot.org/story/11/02/02/1433210/Attacks-Targeting-Classified-Ad-Sites-Surge [slashdot.org]


Hackers Respond To Help Wanted Ads With Malware:

http://it.slashdot.org/story/11/01/20/0228258/Hackers-Respond-To-Help-Wanted-Ads-With-Malware [slashdot.org]


Hackers Use Banner Ads on Major Sites to Hijack Your PC:

http://www.wired.com/techbiz/media/news/2007/11/doubleclick [wired.com]


Ruskie gang hijacks Microsoft network to push penis pills:

http://www.theregister.co.uk/2010/10/12/microsoft_ips_hijacked/ [theregister.co.uk]


Major ISPs Injecting Ads, Vulnerabilities Into Web:

http://it.slashdot.org/it/08/04/19/2148215.shtml [slashdot.org]


Two Major Ad Networks Found Serving Malware:

http://tech.slashdot.org/story/10/12/13/0128249/Two-Major-Ad-Networks-Found-Serving-Malware [slashdot.org]



http://it.slashdot.org/story/09/06/15/2056219/The-Next-Ad-You-Click-May-Be-a-Virus [slashdot.org]



http://news.slashdot.org/article.pl?sid=09/09/13/2346229 [slashdot.org]



http://apcmag.com/microsoft_apologises_for_serving_malware.htm [apcmag.com]


ISP's INJECTING ADS AND ERRORS INTO THE WEB: -> http://it.slashdot.org/it/08/04/19/2148215.shtml [slashdot.org]


ADOBE FLASH ADS INJECTING MALWARE INTO THE NET: http://it.slashdot.org/article.pl?sid=08/08/20/0029220&from=rss [slashdot.org]


London Stock Exchange Web Site Serving Malware:

http://www.securityweek.com/london-stock-exchange-web-site-serving-malware [securityweek.com]


Spotify splattered with malware-tainted ads:

http://www.theregister.co.uk/2011/03/25/spotify_malvertisement_attack/ [theregister.co.uk]


As my list "multiple evidences thereof" as to adbanners & viruses + the fact they slow you down & cost you more (from reputable & reliable sources no less)).

17.) Per point #16, a way to save some money: ANDROID phones can also use the HOSTS FILE TO KEEP DOWN BILLABLE TIME ONLINE, vs. adbanners or malware such as this:


Infected Androids Run Up Big Texting Bills:

http://it.slashdot.org/story/11/03/01/0041203/Infected-Androids-Run-Up-Big-Texting-Bills [slashdot.org]


AND, for protection vs. other "botnets" migrating from the PC world, to "smartphones" such as ZITMO (a ZEUS botnet variant):

http://www.google.com/search?hl=en&source=hp&q=ZITMO&btnG=Google+Search [google.com]


It's easily done too, via the ADB dev. tool, & mounting ANDROID OS' system mountpoint for system/etc as READ + WRITE/ADMIN-ROOT PERMISSIONS, then copying your new custom HOSTS over the old one using ADB PULL/ADB PUSH to do so (otherwise ANDROID complains of "this file cannot be overwritten on production models of this Operating System", or something very along those lines - this way gets you around that annoyance along with you possibly having to clear some space there yourself if you packed it with things!).

18.) Bad news: ADBLOCK CAN BE DETECTED FOR: See here on that note -> http://arstechnica.com/business/news/2010/03/why-ad-blocking-is-devastating-to-the-sites-you-love.ars [arstechnica.com]

HOSTS files are NOT BLOCKABLE by websites, as was tried on users by ARSTECHNICA (and it worked, proving HOSTS files are a better solution for this because they cannot be blocked & detected for, in that manner), to that websites' users' dismay:



An experiment gone wrong - By Ken Fisher | Last updated March 6, 2010 11:11 AM

http://arstechnica.com/business/news/2010/03/why-ad-blocking-is-devastating-to-the-sites-you-love.ars [arstechnica.com]

"Starting late Friday afternoon we conducted a 12 hour experiment to see if it would be possible to simply make content disappear for visitors who were using a very popular ad blocking tool. Technologically, it was a success in that it worked. Ad blockers, and only ad blockers, couldn't see our content."


"Our experiment is over, and we're glad we did it because it led to us learning that we needed to communicate our point of view every once in a while. Sure, some people told us we deserved to die in a fire. But that's the Internet!"

Thus, as you can see? Well - THAT all "went over like a lead balloon" with their users in other words, because Arstechnica was forced to change it back to the old way where ADBLOCK still could work to do its job (REDDIT however, has not, for example). However/Again - this is proof that HOSTS files can still do the job, blocking potentially malscripted ads (or ads in general because they slow you down) vs. adblockers like ADBLOCK!


19.) Even WIKILEAKS "favors" blacklists (because they work, and HOSTS can be a blacklist vs. known BAD sites/servers/domain-host names):


PERTINENT QUOTE/EXCERPT (from -> http://www.theregister.co.uk/2010/12/16/wikileaks_mirror_malware_warning_row/ [theregister.co.uk] )

"we are in favour of 'Blacklists', be it for mail servers or websites, they have to be compiled with care... Fortunately, more responsible blacklists, like stopbadware.org (which protects the Firefox browser)...


20.) AND, LASTLY? SINCE MALWARE GENERALLY HAS TO OPERATE ON WHAT YOU YOURSELF CAN DO (running as limited class/least privlege user, hopefully, OR even as ADMIN/ROOT/SUPERUSER)? HOSTS "LOCK IN" malware too, vs. communicating "back to mama" for orders (provided they have name servers + C&C botnet servers listed in them, blocked off in your HOSTS that is) - you might think they use a hardcoded IP, which IS possible, but generally they do not & RECYCLE domain/host names they own (such as has been seen with the RBN (Russian Business Network) lately though it was considered "dead", other malwares are using its domains/hostnames now, & this? This stops that cold, too - Bonus!)...

Still - It's a GOOD idea to layer in the usage of BOTH browser addons for security like adblock ( http://adblockplus.org/en/ [adblockplus.org] ), IE 9's new TPL's ( http://ie.microsoft.com/testdrive/Browser/TrackingProtectionLists/ [microsoft.com] ), &/or NoScript ( http://noscript.net/ [noscript.net] especially this one, as it covers what HOSTS files can't in javascript which is the main deliverer of MOST attacks online & SECUNIA.COM can verify this for anyone really by looking @ the past few years of attacks nowadays), for the concept of "layered security"....

It's just that HOSTS files offer you a LOT MORE gains than Adblock ( http://adblockplus.org/en/ [adblockplus.org] ) does alone (as hosts do things adblock just plain cannot & on more programs, for more speed, security, and "stealth" to a degree even), and it corrects problems in DNS (as shown above via hardcodes of your favorite sites into your HOSTS file, and more (such as avoiding DNS request logs)).

ALSO - Some more notes on DNS servers & their problems, very recent + ongoing ones:

BIND vs. what the Chinese are doing to DNS lately? See here:

http://yro.slashdot.org/story/10/11/29/1755230/Chinese-DNS-Tampering-a-Real-Threat-To-Outsiders [slashdot.org]



http://www.theregister.co.uk/2010/11/26/secunia_back_from_dns_hack/ [theregister.co.uk]

(Yes, even "security pros" are helpless vs. DNS problems in code bugs OR redirect DNS poisoning issues, & they can only try to "set the DNS record straight" & then, they still have to wait for corrected DNS info. to propogate across all subordinate DNS servers too - lagtime in which folks DO get "abused" in mind you!)


DNS vs. the "Kaminsky DNS flaw", here (and even MORE problems in DNS than just that):

http://www.scmagazineus.com/new-bind-9-dns-flaw-is-worse-than-kaminskys/article/140872/ [scmagazineus.com]

(Seems others are saying that some NEW "Bind9 flaw" is worse than the Kaminsky flaw ALONE, up there, mind you... probably corrected (hopefully), but it shows yet again, DNS hassles (DNS redirect/DNS poisoning) being exploited!)


Moxie Marlinspike's found others (0 hack) as well...

Nope... "layered security" truly IS the "way to go" - hacker/cracker types know it, & they do NOT want the rest of us knowing it too!...

(So until DNSSEC takes "widespread adoption"? HOSTS are your answer vs. such types of attack, because the 1st thing your system refers to, by default, IS your HOSTS file (over say, DNS server usage). There are decent DNS servers though, such as OpenDNS, ScrubIT, or even NORTON DNS (more on each specifically below), & because I cannot "cache the entire internet" in a HOSTS file? I opt to use those, because I have to (& OpenDNS has been noted to "fix immediately", per the Kaminsky flaw, in fact... just as a sort of reference to how WELL they are maintained really!)


DNS provider decked by DDoS dastards:

http://www.theregister.co.uk/2010/11/16/ddos_on_dns_firm/ [theregister.co.uk]


Ten Percent of DNS Servers Still Vulnerable: (so much for "conscientious patching", eh? Many DNS providers weren't patching when they had to!)

http://it.slashdot.org/it/05/08/04/1525235.shtml?tid=172&tid=95&tid=218 [slashdot.org]


DDoS Attacks Via DNS Recursion:

http://it.slashdot.org/it/06/03/16/1658209.shtml [slashdot.org]



http://it.slashdot.org/it/07/02/06/2238225.shtml [slashdot.org]


TimeWarner DNS Hijacking:

http://tech.slashdot.org/article.pl?sid=07/07/23/2140208 [slashdot.org]


DNS Re-Binding Attacks:

http://crypto.stanford.edu/dns/ [stanford.edu]


DNS Server Survey Reveals Mixed Security Picture:

http://it.slashdot.org/it/07/11/21/0315239.shtml [slashdot.org]


Photobucket's DNS records hijacked by Turkish hacking group:

http://www.zdnet.com/blog/security/title/1285 [zdnet.com]


Halvar figured out super-secret DNS vulnerability:

http://www.zdnet.com/blog/security/has-halvar-figured-out-super-secret-dns-vulnerability/1520 [zdnet.com]


BIND Still Susceptible To DNS Cache Poisoning:

http://tech.slashdot.org/tech/08/08/09/123222.shtml [slashdot.org]


DNS Poisoning Hits One of China's Biggest ISPs:

http://it.slashdot.org/it/08/08/21/2343250.shtml [slashdot.org]


HOWEVER - Some DNS servers are "really good stuff" vs. phishing, known bad sites/servers/hosts-domains that serve up malware-in-general & malicious scripting, botnet C&C servers, & more, such as:

Norton DNS -> http://nortondns.com/ [nortondns.com]
ScrubIT DNS -> http://www.scrubit.com/ [scrubit.com]
OpenDNS -> http://www.opendns.com/ [opendns.com]

(Norton DNS in particular, is exclusively for blocking out malware, for those of you that are security-conscious. ScrubIT filters pr0n material too, but does the same, & OpenDNS does phishing protection. Each page lists how & why they work, & why they do so. Norton DNS can even show you its exceptions lists, plus user reviews & removal procedures requests, AND growth stats (every 1/2 hour or so) here -> http://safeweb.norton.com/buzz [norton.com] so, that ought to "take care of the naysayers" on removal requests, &/or methods used plus updates frequency etc./et al...)

HOWEVER - There's ONLY 1 WEAKNESS TO ANY network defense, including HOSTS files (vs. host-domain name based threats) & firewalls (hardware router type OR software type, vs. IP address based threats): Human beings, & they not being 'disciplined' about the indiscriminate usage of javascript (the main "harbinger of doom" out there today online), OR, what they download for example... & there is NOTHING I can do about that! (Per Dr. Manhattan of "The Watchmen", ala -> "I can change almost anything, but I can't change human nature")

HOWEVER AGAIN - That's where NORTON DNS, OpenDNS, &/or ScrubIT DNS help!

(Especially for noob/grandma level users who are unaware of how to secure themselves in fact, per a guide like mine noted above that uses "layered-security" principles!)

ScrubIT DNS, &/or OpenDNS are others alongside Norton DNS (adding on phishing protection too) as well!

( & it's possible to use ALL THREE in your hardware NAT routers, and, in your Local Area Connection DNS properties in Windows, for again, "Layered Security" too)...




"Ever since I've installed a host file (http://www.mvps.org/winhelp2002/hosts.htm) to redirect advertisers to my loopback, I haven't had any malware, spyware, or adware issues. I first started using the host file 5 years ago." - by TestedDoughnut (1324447) on Monday December 13, @12:18AM (#34532122)

"I also use the MVPS ad blocking hosts file." - by Rick17JJ (744063) on Wednesday January 19, @03:04PM (#34931482)

"I use ad-Block and a hostfile" - by Ol Olsoc (1175323) on Tuesday March 01, @10:11AM (#35346902)

"^^ One of the many reasons why I like the user-friendliness of the /etc/hosts file." - by lennier1 (264730) on Saturday March 05, @09:26PM (#35393448)

"I use a custom /etc/hosts to block ads... my file gets parsed basically instantly ... So basically, for any modern computer, it has zero visible impact. And even if it took, say, a second to parse, that would be more than offset by the MANY seconds saved by not downloading and rendering ads. I have noticed NO ill effects from running a custom /etc/hosts file for the last several years. And as a matter of fact I DO run http servers on my computers and I've never had an /etc/hosts-related problem... it FUCKING WORKS and makes my life better overall." - by sootman (158191) on Monday July 13 2009, @11:47AM (#28677363) Homepage Journal

"I do use Hosts, for a couple fake domains I use." - by icebraining (1313345) on Saturday December 11, @09:34AM (#34523012) Homepage

"They've been on my HOSTS block for years" - by ScottCooperDotNet (929575) on Thursday August 05 2010, @01:52AM (#33147212)

"Better than an ad blocker, imo. Hosts file entries: http://www.mvps.org/winhelp2002/hosts.htm [mvps.org] [mvps.org]" - by TempestRose (1187397) on Tuesday March 15, @12:53PM (#35493274)

"I'm currently only using my hosts file to block pheedo ads from showing up in my RSS feeds and causing them to take forever to load. Regardless of its original intent, it's still a valid tool, when used judiciously." - by Bill Dog (726542) on Monday April 25, @02:16AM (#35927050) Homepage Journal

"you're right about hosts files" - by drinkypoo (153816) on Thursday May 26, @01:21PM (#36252958) Homepage

"put in your /etc/hosts:" - by Anonymous Coward on Friday December 03, @09:17AM (#34429688)


Then, there is also the words of respected security expert, Mr. Oliver Day, from SECUNIA.COM to "top that all off" as well:


http://www.securityfocus.com/columnists/491 [securityfocus.com]

Some "PERTINENT QUOTES/EXCERPTS" to back up my points with (for starters):


"The host file on my day-to-day laptop is now over 16,000 lines long. Accessing the Internet -- particularly browsing the Web -- is actually faster now."

Speed, and security, is the gain... others like Mr. Day note it as well!


"From what I have seen in my research, major efforts to share lists of unwanted hosts began gaining serious momentum earlier this decade. The most popular appear to have started as a means to block advertising and as a way to avoid being tracked by sites that use cookies to gather data on the user across Web properties. More recently, projects like Spybot Search and Destroy offer lists of known malicious servers to add a layer of defense against trojans and other forms of malware."

Per my points exactly, no less... & guess who was posting about HOSTS files a 14++ yrs. or more back & Mr. Day was reading & now using? Yours truly (& this is one of the later ones, from 2001 http://www.furtherleft.net/computer.htm [furtherleft.net] (but the example HOSTS file with my initials in it is FAR older, circa 1998 or so) or thereabouts, and referred to later by a pal of mine who moderates NTCompatible.com (where I posted on HOSTS for YEARS (1997 onwards)) -> http://www.ntcompatible.com/thread28597-1.html [ntcompatible.com] !


"Shared host files could be beneficial for other groups as well. Human rights groups have sought after block resistant technologies for quite some time. The GoDaddy debacle with NMap creator Fyodor (corrected) showed a particularly vicious blocking mechanism using DNS registrars. Once a registrar pulls a website from its records, the world ceases to have an effective way to find it. Shared host files could provide a DNS-proof method of reaching sites, not to mention removing an additional vector of detection if anyone were trying to monitor the use of subversive sites. One of the known weaknesses of the Tor system, for example, is direct DNS requests by applications not configured to route such requests through Tor's network."

There you go: AND, it also works vs. the "KAMINSKY DNS FLAW" & DNS poisoning/redirect attacks, for redirectable weaknesses in DNS servers (non DNSSEC type, & set into recursive mode especially) and also in the TOR system as well (that lends itself to anonymous proxy usage weaknesses I noted above also) and, you'll get to sites you want to, even IF a DNS registrar drops said websites from its tables as shown here Beating Censorship By Routing Around DNS -> http://yro.slashdot.org/story/10/12/09/1840246/Beating-Censorship-By-Routing-Around-DNS [slashdot.org] & even DNSBL also (DNS Block Lists) -> http://en.wikipedia.org/wiki/DNSBL [wikipedia.org] as well - DOUBLE-BONUS!


* POSTS ABOUT HOSTS FILES I DID on "/." THAT HAVE DONE WELL BY OTHERS & WERE RATED HIGHLY, 15++ THUSFAR (from +3 -> +1 RATINGS, usually "informative" or "interesting" etc./et al):

HOSTS MOD UP -> http://yro.slashdot.org/comments.pl?sid=1907266&cid=34529608 [slashdot.org]
HOSTS MOD UP -> http://tech.slashdot.org/comments.pl?sid=1490078&cid=30555632 [slashdot.org]
HOSTS MOD UP -> http://it.slashdot.org/comments.pl?sid=1869638&cid=34237268 [slashdot.org]
HOSTS MOD UP -> http://tech.slashdot.org/comments.pl?sid=1461288&threshold=-1&commentsort=0&mode=thread&cid=30272074 [slashdot.org]
HOSTS MOD UP -> http://tech.slashdot.org/comments.pl?sid=1255487&cid=28197285 [slashdot.org]
HOSTS MOD UP -> http://tech.slashdot.org/comments.pl?sid=1206409&cid=27661983 [slashdot.org]
HOSTS MOD UP -> http://apple.slashdot.org/comments.pl?sid=1725068&cid=32960808 [slashdot.org]
HOSTS MOD UP -> http://it.slashdot.org/comments.pl?sid=1743902&cid=33147274 [slashdot.org]
HOSTS MOD UP -> http://news.slashdot.org/comments.pl?sid=1913212&cid=34576182 [slashdot.org]
HOSTS MOD UP with facebook known bad sites blocked -> http://tech.slashdot.org/comments.pl?sid=1924892&cid=34670128 [slashdot.org]
HOSTS FILE MOD UP FOR ANDROID MALWARE -> http://mobile.slashdot.org/comments.pl?sid=1930156&cid=34713952 [slashdot.org]
HOSTS MOD UP ZEUSTRACKER -> http://it.slashdot.org/comments.pl?sid=2059420&cid=35654066 [slashdot.org]
HOSTS MOD UP vs AT&T BANDWIDTH CAP -> http://tech.slashdot.org/comments.pl?sid=2116504&cid=35985584 [slashdot.org]
HOSTS MOD UP CAN DO SAME AS THE "CloudFlare" Server-Side service -> http://it.slashdot.org/comments.pl?sid=2220314&cid=36372850 [slashdot.org]
HOSTS and BGP +5 RATED (BEING HONEST) http://tech.slashdot.org/comments.pl?sid=1901826&cid=34490450 [slashdot.org]


* "Here endeth the lesson..."


P.S.=> SOME MINOR "CAVEATS/CATCH-22's" - things to be aware of for "layered security" + HOSTS file performance - easily overcome, or not a problem at all:

A.) HOSTS files don't function under PROXY SERVERS (except for Proximitron, which has a filter that allows it) - Which is *the "WHY"* of why I state in my "P.S." section below to use both AdBlock type browser addon methods (or even built-in block lists browsers have such as Opera's URLFILTER.INI file, & FireFox has such as list as does IE also in the form of TPL (tracking protection lists -> http://ie.microsoft.com/testdrive/Browser/TrackingProtectionLists/ [microsoft.com] , good stuff )) in combination with HOSTS, for the best in "layered security" (alongside .pac files + custom cascading style sheets that can filter off various tags such as scripts or ads etc.) - but proxies, especially "HIGHLY ANONYMOUS" types, generally slow you down to a CRAWL online (& personally, I cannot see using proxies "for the good" typically - as they allow "truly anonymous posting" & have bugs (such as TOR has been shown to have & be "bypassable/traceable" via its "onion routing" methods)).

B.) HOSTS files do NOT protect you vs. javascript (this only holds true IF you don't already have a bad site blocked out in your HOSTS file though, & the list of sites where you can obtain such lists to add to your HOSTS are above (& updated daily in many of them)).

C.) HOSTS files (relatively "largish ones") require you to turn off Windows' native "DNS local client cache service" (which has a problem in that it's designed with a non-redimensionable/resizeable list, array, or queue (DNS data loads into a C/C++ structure actually/afaik, which IS a form of array)) - mvps.org covers that in detail and how to easily do this in Windows (this is NOT a problem in Linux, & it's 1 thing I will give Linux over Windows, hands-down). Relatively "smallish" HOSTS files don't have this problem (mvps.org offers 2 types for this).

D.) HOSTS files, once read/loaded, once GET CACHED, for speed of access/re-access (@ system startup in older MS OS' like 2000, or, upon a users' 1st request that's "Webbound" via say, a webbrowser) gets read into either the DNS local caching client service (noted above), OR, if that's turned off? Into your local diskcache (like ANY file is), so it reads F A S T upon re-reads/subsequent reads (until it's changed in %WinDir%\system32\drivers\etc on Windows, which marks it "Dirty" & then it gets re-read + reloaded into the local diskcache again). This may cause a SMALL lag upon reload though, depending on the size of your HOSTS file.

E.) HOSTS files don't protect vs. BGP exploits - Sorry, once it's out of your hands/machine + past any interior network + routers you have, the packets you send are out there into the ISP/BSP's hands - they're "the Agents" holding all the keys to the doorways at that point (hosts are just a forcefield-filter (for lack of a better description) armor on what can come in mostly, & a bit of what can go out too (per point #20 above on "locking in malware")). Hosts work as a "I can't get burned if I can't go into the kitchen" protection, for you: Not your ISP/BSP. It doesn't extend to them

F.) HOSTS files don't protect vs. IP addressed adbanners (rare) &/or IP address utilizing malwares (rare too, most used domain/host names because they're "RECYCLABLE/REUSEABLE"), so here, you must couple HOSTS files w/ firewall rules tables (either in software firewalls OR router firewall rules table lists)... apk

Check for New Comments
Slashdot Login

Need an Account?

Forgot your password?