Criteria for Ratings - Effective January 2006
Ratings are based on 7 categories. Top value of a category can range from 5 to 30 points. Within each category, a score of from 0 to maximum is assigned using the descriptions below. If a guidebook isn't exactly covered by a description, the score was assigned based on closest approximation. When scores for all guidebooks are assigned, they are ranked in numeric order. Preliminarily, stars for 2005 will be assigned based on the following scale (raised during 2004):
Preferences: INNSTAR prefers guides that provide detailed information on the inns to prospective inn guests, has sufficient numbers of inns to offer a choice, and is easy to use without the latest browser or high-speed internet connections. Additional features that may help the guest are a plus; features designed to merely attract traffic to the site without adding any service to the prospective guest are not evaluated, even though they may be of value to the innkeeper. A large number of inn names with basic contact information only is not considered to be of value; it is a minus if not well edited for closed inns & duplications. Note: The term State also includes Canadian Provinces.
- 5 Innstars: 80 and above
- 4 Innstars: 65 to 79
- 3 Innstars: 50 to 64
- 2 Innstars: 35 to 49
- 1 Innstar: 20 to 34
- 0 Innstars: under 20
Methodology: Each guide is reviewed using Mozilla Firefox (primary)and WindowsXP; I may also verify using Opera and earlier versions of Netscape and MSIE 6 and Windows operating systems. Guides with search capabilities are evaluated using several amenity combinations. I use a dial-up connection, typically in the 46-48 Kbs range.
Recent Changes: For a summary of important changes, see change summary below.
Here are the Descriptions in each area:
A: Completeness of Coverage for Descriptive Listings (0-20) (a descriptive listing gives details about the Inn, more than contact info and may include links to independent pages) - rated for North America only and adjusted for regional or specialty guides. Note: To ensure a level of consistency, I established some general point ranges based on number of inns covered; I then look at the scale below to come up with the actual score in this area.
B: Depth of Information (0-20) (how much information is offered to help you pick an Inn; caution - a guide may score well in this area but poorly in category 1)
- 20 - Extremely thorough coverage throughout full coverage area, based on approximate inn density for various regions.
- 15 - Numerous inns in much of area, remaining area has some inns.
- 10 - Numerous in limited areas, some gaps in remaining area.
- 5 - Limited number of listings.
- 0 - Listings are basic contact only.
C: Ease of Use - Finding Inns (0-30) (how easy is it to find Inns in specific areas or cities and with specific features/amenities.
- 20 - Descriptive listings have several pictures, details on rooms, amenities, rates, reservation policies, payment methods, local area info. Has weather and map links. Listings include contact info, innkeeper names, rates, payment methods. Most listings have highly visible links to independent Inn websites.
- 15 - Descriptive have at least 1 picture (often more), detailed description of inn & amenities.
All listings have contact info, & rates; many also have links to independent Inn websites.
- 10 - Some descriptive listings do not have pictures. All listings have detailed description of inn, amenities, rates.
- 8 - Max score for 'links only' guide without other descriptive data.
- 5 - Descriptive listing is text with moderately detailed description, rates, reservation info with few or no pictures.
- 0 - List consists of basic contact information only or insufficient data.
D: Ease of Use - web pages (0-10) (is system logical, are graphics sized and organized reasonably, are hot buttons or links easy to find and labeled clearly, is there excessive scrolling or must one wait for screen after screen to be retrieved, are page and graphics sizes/numbers reasonable for dial-up connections?)
- 30 - Has multiple effective ways of locating inns. Must be able to locate likely inns in a specific area without requiring detailed geographic knowledge. Detailed maps available for searching/finding B&Bs. Also available by specific city or specific Inn. Includes ability to look or select for multiple amenities, features, restrictions or qualities.
- 25 - Adept at locating likely inns in a state or area of state without detailed geographic knowledge. Maps may be included. Also available by specific city or specific Inn. Includes ability to look for most popular amenities, features, restrictions or qualities.
- 20 - Has either good geographical location ability or thorough amenities/features search.
- 15 - May look by state, city or Inn with some knowledge of area needed. Includes either limited ability to look for features or to narrow area geographically.
- 10 - Listed by city. Must know cities to look for region within state.
- 5 - Inns listed alphabetically by Inn name, making it difficult to find Inns in a general area; or must look at all cities to find which have detailed listings.
- 2 - Inn and/or city lists, if available, are not logically organized for common use.
- 0 - Insufficient clients to determine usability of list.
- 10 - Extremely convenient, loading time quick even when loading all images. Excellent balance of scrolling vs multiple screens. Screens easy to use and logical. Search engine, if used, is fast.
- 7 - Convenient, quick loading. Screens logical but data not available as quickly (image quantity, size or format; number of pages required). However, 7 is max score for guide optimized for a specific browser.
- 5 - Moderately convenient. Some delays in getting to Inn info because of multiple screens or long scrolling. May not work properly using older browsers. Some design aspects may make you look for buttons.
- 3 - Fairly slow or inconvenient due to slow search engines or too many screens or graphics. Have to think about how to find inn lists.
- 1 - Inconvenient and slow. Typical browser would click off from frustration.
- 0 - Does not have enough Inns to properly evaluate
E: Quality Assurance (0-5) (does guide indicate inns that are inspected by guide writer or by recognized agencies, have a supervised system for providing guest feedback, or inns are recommended by guests for inclusion) Criteria value downgraded based lack of industry-wide standards.
F: On-Line Availability or Reservations (0-10) (does guide have ability, either native to site or through external links, to provide on-line availability and/or secure reservations?) Criteria value upgraded based on tracking experience.
- 5 - All Inns listed are personally visited by guide author, or all Inns listed are inspected by nationally-recognized service. Has supervised system for guest feedback to author. Descriptions either written by author or, if Inn-provided, implicitly verified by inspecting agency.
- 3 - Clearly indicates which inns are inspected and identifies inspecting organization. May also have guest feedback system.
- 2 - Indicates inns that are inspected but does not indicate organization or has guest feedback system.
- 1 - Indicates affiliation or memberships.
- 0 - No obvious provision for quality asssurance.
G: Site-provided extras (0-5) (does guide have added-value extras that enhance its usefulness to potential guests, including, in order of perceived value)
- 10 - Has on-line availability for a substantial number of inns using widely-used programs.
- 6 - Has on-line availability using own proprietary system only.
- 1 - Has on-line reservation request form only.
- 0 - Option not available.
1. Promotes last minute specials.
2. Has information about B&Bs in general.
3. Newsletter, general bulletin boards or On-line magazine
4. Tourism links, additional regional information
5. Recipes, association lists, specialty market lists, etc.
- 1-5 - Subjective evaluation, based on number & value of extras.
Z1: Special Problems: Points may be deducted based on such problems as failure to correct errors timely, misleading data or description of the service, hard-to-read wallpaper, etc. These are progressive; if the same errors persist from year to year, the points deducted will increase.
Summary of Changes for 2006
- Inn Numbers (A) criteria applied somewhat differently with a more even spread between guides based onnumbers.
- Quality Control (E) deemphasized due to lack of nationally-recognized standards group and growth of external hotel rating systems.
- On-Line Availability (F) reemphasized based on experience with guest usage patterns.
- Primary browser changed in methodology