Scraping data from MP (or how about a public API?)

Since MP doesn't have a public API (as say The Crag: thecrag.com/article/api ), what are the rules to scraping the data?

In the Terms and Conditions ( mountainproject.com/scripts…):

Ownership
As between you and THE COMPANY, you own Your Content. We own the THE COMPANY Content, including but not limited to visual interfaces, interactive features, graphics, design, compilation, computer code, products, software, aggregate user reviews or ratings, and all other elements and components of the Service excluding Your Content, User Content and Third Party Content.

This implies that MP takes no ownership over user submitted content.

Say I wanted to create a third party app that scrapes the data of user submitted content such as beta and photos (and attribute credit to the original user). Is that kosher?

No, it is not kosher - for many reasons, not least of which is you would be violating the copyright of all the submitted content. The content was submitted to MP, not another site and the TOS is between us and the user. By displaying the content on a third party site/app you are violating the original copyright holder unless you receive explicit permission from each contributor.

So what is the TOS between a third party and MP? Does one even exist? If one exists, please elaborate the difference between profit vs. non-profit.

"By displaying the content on a third party site/app you are violating the original copyright holder unless you receive explicit permission from each contributor."

Why wouldn't attributing credit to a contributor qualify?

Contributed by Colonel Mustard. I like it.

Eric Thomas wrote:So what is the TOS between a third party and MP? Does one even exist? If one exists, please elaborate the difference between profit vs. non-profit. "By displaying the content on a third party site/app you are violating the original copyright holder unless you receive explicit permission from each contributor." Why wouldn't attributing credit to a contributor qualify?

"Receive" and "attribute" mean vastly different things. This first one's free...after that I bill my standard rate of $400 an hour. Andy, of course, gets all the free legal consulting he wants.

Because the contributing users have not given you explicit permission to use their content within your application/site.

If I steal someone's photograph from one book and put it into another book, just displaying the original photographers name does not solve the fact that I did not get permission to copy that photograph into a new book.

Eric Thomas wrote:So what is the TOS between a third party and MP? Does one even exist? If one exists, please elaborate the difference between profit vs. non-profit. "By displaying the content on a third party site/app you are violating the original copyright holder unless you receive explicit permission from each contributor." Why wouldn't attributing credit to a contributor qualify?

I don't really have anything to add beyond what Andy's said, but feel compelled to chime in.

Scraping content from MP (and displaying elsewhere) is a blatant copyright violation, end of story. You would be violating the copyright of the original contributor, and possibly Mountain Project as well, depending on what was copied.

Because copyright is the general law, I don't think our TOS come into play here.

Fair enough. I probably should have been more explicit with my request, but I was interested in an API to build off of (requested here with no response: mountainproject.com/v/mount… )

It's a common idiom these days for large web sites to provide an API (with limited and unlimited usage).
e.g.
dailymile.com/api
yelp.com/developers/documen…
developers.facebook.com/doc…

These websites are displaying user content for third parties to develop off of. Their TOS states how you can use this content. I'm sure there's something in there that states you can't take the content and publish it into a book. And more importantly, I'm fairly sure whatever is built must be not for profit.

In essence, I figured since it appears MP has no interest in providing a public API -- I had to ask if there was another means of building something off of it. Consider it a compliment that people want to take their free time away from climbing to extend the MP ecosystem.

Sorry for the hubbub. Carry on.

Just DON'T do it!!!!

since when is this not a 'public' app?
mountainproject.com/mobileApps

am I a member of an elite private club that I didn't know about?

John Marsella wrote:My app will replace all the beta pictures with cat pictures and probably rename routes cat-themed things. I will call it MEOWntainProject. Is that halal?

Best damn post I've seen in a long time.

Andy Laakmann wrote: If I steal someone's photograph from one book and put it into another book, just displaying the original photographers name does not solve the fact that I did not get permission to copy that photograph into a new book.

There seems to be some type of rule differentiations between people who are reporting about a subject (e.g. college students, news reporters, ect.) and someone who is using an external source for direct gain.

For example, if I wanted to write a research paper on the growth of Internet-based climbing forums, I would probably pull statistical information about, and from, this website. As so long as I cite the information in accordance with APA guidelines, no professor would have a problem with my essay, and I dont need permission to cite in my paper information I learn from MP.com.

Let me provide a more-realistic example of a blurb a college student might include in a paper.

Copyright wars rage on among website administrators and website users. For example, a recent thread on the Mountain Project website sparked an inter-site debate between users and administrators about how user rights relate to copying information. Website user Eric Thomas believes that the Mountain Project website does not take ownership of the information on its website (Thomas, 2013). But website administrator, Andy Laakmann (2013), said "The content was submitted to MP, not another site and the TOS is between us and the user." (para 1).

Reference

Thomas, E. (2013). Scraping data from MP (or how about a public API?). Retrieved from mountainproject.com/scripts…

Laakmann, A. (2013). Scraping data from MP (or how about a public API?). Retrieved from mountainproject.com/scripts…

I could write that information in a research paper and copyright law does not prevent me from doing so. Where is the line drawn then?

John Marsella wrote:My app will replace all the beta pictures with cat pictures and probably rename routes cat-themed things. I will call it MEOWntainProject. Is that halal?

For the win!!!

But remember, phone books are not copyright-able.

So the contributor owns their description of a route, and any pictures of it, but they do not own the fact that the route exists: one may look up names and locations of routes on MP to their heart's content, and even publish the information they glean from that.

Just like I owe the phone company nothing by looking up phone numbers in their book, even massive amounts of them, and doing what I like with the data. Unless something changed, that's a long settled area of copyright.

Seeing as how MP is a publicly generated resource, I think this is where people are coming from when it comes to asking for an API.

Since we all contributed to the database in little ways.

It's interesting that MP is a mix of original, copyright-able data, and mundane, simple-collections-of-facts data that is not copyright-able.

It would be nice to have an API to work with. Many people who have responded to this thread obviously do not understand what that means. It doesn't mean that someone is going to make a rival site or even try to make money off of the data in MP.com.

As an example, I would like to build my own database for my own personal tick list. There are many things about the way MP.com uses it's data and forces upon the user that I personally do not like. I would like to improve those things and also add additional features, like charting. All of this would be possible with a public API, and would not harm anyone. In fact, it would make MP.com move valuable as a resource.

We actually do now have a rudimentary API. It's primary use is to retrieve personal tick data. Because it is new, and I may change it (and break your code that uses it), contact me if you'd like to give it a try.

That's awesome. Thanks for replying. I will message you to get more info.

Thanks!

Great! I would be very interested in an API to gain access to my ToDo routes. Ideally JSON returns with area location coordinates but I realize you are just getting started with this. These are steps in a great direction!

I understand that MP already took a stance on their Terms of Use but I would like to direct some consideration toward how Open Street Map opens their crowd sourced data. For those that don't know, Open Street Map is a crowd sourced map of the world with thousands of active contributors and many more users. They have a completely open data policy which means that as a contributor I in turn have the benefit of being able to use the data I submit along with all the rest of the map data. This is a huge benefit to contributors as the core app provides a specific service but the data can be extracted for additional niche purposes. This has resulted in a host of customized web maps being offered by agencies like MapQuest as well as empowering smaller groups to build data on an existing data storage option that almost immediately benefits them as a service. The best example of this is with the humanitarian group HOT OSM which goes to impacted areas like Heiti after the earth quake and partners with locals to rebuild a map of the areas destroyed to help with rebuilding. MP isn't mapping such critical data but can still be very useful for all kinds of applications including as a potential proxy measurement for health analytics. In an ideal world I think this kind of model is what is needed from sites like MP and I hope they move toward this model.

openstreetmap.org/copyright

This thread is pretty outdated, but I wanted to ask - does it violate the terms of service to copy only "public information" (areas, routes, and basic info about each such as type, grade, etc)? From what it sounds like, the main problem the Admins are stating is to do with user-submitted content (comments, pictures, possibly even route descriptions)?

If I was to create an application where a user asks for information about a route by search query and I provide a snippet of that route's information (maybe type, grade, location, etc) with a link to the route's page on MountainProject, is this acceptable?

dantesupertramp wrote:

This thread is pretty outdated, but I wanted to ask - does it violate the terms of service to copy only "public information" (areas, routes, and basic info about each such as type, grade, etc)? From what it sounds like, the main problem the Admins are stating is to do with user-submitted content (comments, pictures, possibly even route descriptions)?

If I was to create an application where a user asks for information about a route by search query and I provide a snippet of that route's information (maybe type, grade, location, etc) with a link to the route's page on MountainProject, is this acceptable?

I'm also curious about this. As the person above me said, it seems that the that "public information" on MP as stated above (minimally speaking the route name and location), is not subject to copywrite from what I understand. Presumably it would be OK to scrape this data from MP into a personal project? The new API is nice, but does not allow one to get route IDs (for instance by crag, by state, etc.) Maybe that is what the lat/long method is for, but I haven't checked it out yet.

Thanks