Data-Mining for Astronomy

I admit it. I’m not on facebook. And I’ve been holding off tweeting as well. Sometimes I think the associations with all things facebook and twitter is like a virus. Do I really want to follow brand X on facebook? Really? But I’ve decided Twitter in particular can serve a very useful role in bringing attention to news, events, research papers, and web-sites dedicated to astronomical data-mining. I can’t tell you how many times I’ve come across a story that I wanted to share on this site but didn’t really have the time to write about it. So I’m going to continue to try to make AstroDataMining.net a useful resource for those interested in learning about this emerging new field. I think there is a significant opportunity and need for more fellow computer scientists to become involved in the field. But at the same time, I’ve started a twitter account to address those frequent occasions when some new development occurs in Astronomical Data Mining. SO… Follow Data-Mining for Astronomy on Twitter!

After something of a long hiatus, I have returned in recent months to writing about astronomical data-mining.    My goal has always been to make this site more than a collection of random thoughts, but also a valuable public resource.   Along those line, I’ve recently made a few changes.

First of all, I gave the site it’s own domain: astrodatamining.net. This is something you can do as an upgrade to a wordpress blog (for $15 / year).    I hope the new site domain will aid in keeping the site at the top of the search engine listings.

Today I created a first cut at a resources page.    Here you will find links to astronomical catalog repositories, data-mining and visualization tools, and database systems.   I am focusing on free open-source technologies because one of the goals of this site has always been to get the amateur astronomy community more involved in data analysis and data-mining, and also because, happily I can!   Some of the best web-sites and software packages available are freely available.    But the resources page is still very much a work in progress, and I would very much welcome suggestions.   If you have a software tool or website that you have found to be particularly useful in your astronomical research, whether you are a professional or amateur, please let me know.

I have been trying to broaden the types of articles that will be presented on the site homepage.    I still plan to continue to have both longer “Perspective” pieces where I try to go deeper into the issues or review technologies emerging from this rapidly evolving field.   Additionally I will be posting on a more regular basis news summaries and paper abstracts along with useful pointers to background information that can help to put the research into perspective.

Additionally, as part of my ResearchSynopsis.net initiative, where I am inviting scientists to submit an authoritative synopsis for the lay reader, I am also hoping to include more contributions from professional astronomers who can tell us, in their own words why their latest paper is interesting and scientifically significant, and explain for the average reader some of the techniques and tools they used.   As an example, see the recent piece on finding cataclysmic variable stars contributed by Denis Denisenko of the Space Research Institute in Moscow.  This invitation is also open to users wishing to submit a detailed description of a research paper, a software review, or a tutorial.    I very much hope that this site will become a community of fellow enthusiasts.

For those of you interested in networking with others interested in this field, I have also created a LinkedIn group.    You’ll need to be a LinkedIn member to join, however.

I’m happy to report that these efforts seem to be paying off.    As shown in the figure below, July 2010 saw a record number of visitors to this site, thanks in no small part to the AAVSO kindly mentioning it in their news listings. (Thank you again, Aaron Price).

My thanks to those (few) of you reading this blog.    Spread the word.    It’s tremendously motivating to write for an audience who shares my enthusiasm for what is, afterall, a pretty specialized (but cool!) topic.   Cheers.

Welcome to Data-Mining for Astronomy.   The purpose of this site is to disseminate articles and ideas concerning computational problems in Astronomy, particularly those related to pattern-recognition and data-mining.   Astronomy is experiencing some exciting developments in this field due to the advent of astronomical surveys such as the Sloan Digital Sky Survey (SDSS) and grid computing projects like Einstein@Home.    These kinds of projects may also create exciting opportunities for amateur astronomers with a knack for software development and wishing to make real scientific contributions. In biology, the emergence of high-throughput experimental technologies such as whole-genome sequencing, gene expression analysis, and protein interaction arrays (to name but a few) created an enormous flood of data that was singularly responsible for the development of the field of bioinformatics.   We now see the very same trend beginning to occur in astronomy, and I very much expect that “astroinformatics” will play an increasingly important role in astrophysical research.   I hope that this site will serve as a clearinghouse for the latest developments in this area and even lead to new collaborations between scientists and software developers.    We live in interesting times!